[ANNOUNCE] x-jhbuild 0.2
halsmit at t-online.de
Thu Nov 18 01:35:53 PST 2010
X-JHBuild is a framework for building and working with X.org modules.
It makes use of JHBuild and is specialized for working with modules that
use Git as VCS.
Install/Uninstall easier than ever before:
It is now possible install X-JHBuild with just a few commands.
$> git clone git://x-jhbuild.git.sourceforge.net/gitroot/x-jhbuild/x-jhbuild
$> cd x-jhbuild
$> make install
Private versions of JHBuild and Git-Python will be installed behind the
scenes. There is an uninstall target which will leave nothing but the
configuration in ~/.x-jhbuild and possibly the two standard directories
~/.local/bin and ~/.local/share behind.
More info about installing and using X-JHBuild can be found at in the wiki at:
Early and premium adopters need to migrate the settings to the new
configuration mechanism, and modify/restore existing repository groups.
See below for more.
Maybe Start Over For Exploration?
There are diverse changes which might be best explored with a new
installation and a new repository group. To do that just rename
~/.x-jhbuild, install anew, and create a new repository group with the
The main changes are the new configuration mechanism and persistent
configuration/moduleset selection, both of which are described below and
in help messages of the corresponding plug-ins.
To use an existing repository group with this new version, rename or
remove the old configuration directory (.xjh/config) and execute the
following command within that repository group:
$> xjh init --restore
The module pthread-stubs has been renamed and so its location moved from
xcb/pthread-stubs to xcb/libpthread-stubs. In the future I intend to
provide a plug-in and a special file in the project's moduleset
repository to facilitate syncing with such repository moves. Currently
you would need to execute the following command from the repository root
$> mv -i xcb/pthread-stubs xcb/libpthread-stubs.
Features new in this version:
Persistent Config-Token/Moduleset Selection
The '-f' and '-m' main command line options are no more. The new
plug-in 'checkout' can be used to permanently select a config-token
and/or moduleset to be used. To have greater control over the state of
repositories, the subcommand 'update' does no longer switch to the
states requested in modulesets and buildconfig.branches for checked out
Git repositories. That can be done with the new plug-in 'spool', which
in the future might also be able to backup and restore repository
states. 'update' does still switch revisions when cloning a repository.
Nothing changes for repositories that don't use Git.
This might complicate the one-time test execution, but I think the cases
where one does not want to execute a follow-up command with the same
selection are practically nonexistent. There can be a three-liner
plug-in that does checkout/spool/build, to accomplish the same as before.
The prompt sign has been extended to optionally display the currently
selected configuration and/or moduleset. This can result in a rather
lengthy sign, but note that the repogroup-name, config-token, and
moduleset-spec (moduleset-aliases) are under your control and can be
represented with short mnemonics.
The New Configuration Mechanism:
The settings in the global configuration file ~/.x-jhbuild/xjhrc will be
overridden by the ones in a repository group at .xjh/xjhrc. The result
can be further tweaked by selecting a file in .xjh/xjhrc.token. These
files use an ini style format that is described in the example file at
~/.x-jhbuild/xjhrc.example. The new plug-in 'config' can be used to
display whole configuration sections with accompanying help text. Such
output can be used as template content for a configuration file. The
previously used configuration files are no longer supported and the
settings have to be migrated. The 'config' plug-in can be of great aid
here. All JHBuild settings can be found in the configuration section
'buildconfig'. The following command generates a template section for
the build configuration:
$> xjh config --verbose buildconfig
The plug-in 'diff-ms' can compare two modulesets and, in a special mode,
display settings of one moduleset. The focus lies on location, version,
and build settings. It does not compare dependencies, currently. I am
confident about Autotools and Git, but comparison of other types might
not be completely accurate. Patches are welcome -- it's in the
moduletraits.py library module.
The New Status Plug-In:
The new plug-in 'status' is able to search selected or all Git
repositories for interesting characteristics and will display status
labels and a few log lines, in an attempt to remind about deviations
from a fresh, unmodified checkout. It is possible to search for
specific characteristics like a configured octopus, or commits that are
nowhere upstream. There is an alternative output format that can be
used to further process the information with another script. See 'xjh
status -h' for more.
Environment, Callback and Execution
The callback and runtime-switch environment variables have been
deprecated. Callbacks can now be done with another main command line
option '--callback' and the environment switch is demonstrated in the
plug-in 'show-env' -- copy-paste two instead of one line.
It is possible to run xjh from source by setting the environment
variable XJHBUILD_RUN_FROM_SOURCE_WITH_SCRIPT_FROM_SOURCE to a
non-empty string. In that case, 'xjh' in PATH has to be a symlink to
the script in the sources. An initial installation is necessary, for
the JHBuild and Git-Python parts.
To ensure the callback executes the same script, there is now a check in
the main script that tries to find 'xjh' in $PATH and generates an error
if it cannot be found or is not the same real file.
Instead of implementing a module selection interface in every plug-in,
the standard input stream (by means of 'modinfo', 'get-cwd-module',
'echo', etc) can be used to select modules. Note, that this does
currently not apply to JHBuild subcommands.
'modinfo' does include dependencies by default, now, as I thought that
that would be the common use case on the command line. In scripts the
new, more intuitive option '--one' can be used to get attributes of a
Module Location Changes
The module pthread-stubs has been renamed to libpthread-stubs to comply
with the policy of using package-names for module-ids. To allow reverse
mapping of a repository to its module-id, the location changed from
xcb/pthread-stubs to xcb/libpthread-stubs.
- The plug-in 'init' can now download a project configuration file,
which will then be installed as .xjh/xjhrc. This way projects can
provide default modulesets and locations to download them from.
- Subcommand aliases can now include main command options. Together with
the new configuration mechanism, it is now possible to create
aliases that change with the repository group.
- 'modinfo --deps' does no longer automatically swallow modules
configured to be skipped. There are new command line options (-s|-S)
to include/exclude such modules explicitly.
- Generate error messages if the Git >= 1.7 dependency is not met. The
prompt sign will display an error message instead of the module name,
and the CWD-Module adaption will fail.
- The implicit subcommand has been abolished in favor of explicitness.
- The plug-in 'edit-config' is no more. I recommend writing a small
plug-in that starts your favorite editor with all active configuration
files -- all info for that is available from the environment. See
'show-env' or commit 184ace5.
- The plug-in 'show-config' is no more. The 'config' plug-in combined
with command-aliases is much more flexible.
- Plug-in names may not contain a dot because that is the section
separator for configuration sections. There is the policy, that all
configuration sections in a plug-in should be assembled below the
top-level section that has the same name as the plug-in. This is
necessary to let 'config' find that section.
Internal changes and additions:
- A current version of Git-Python is made available to all plug-ins.
There is the class GitRepoInfoAccess in the library module gitaccess,
which focuses on obtaining information about a repository. Together
with the comments accompanying that class it might be possible to
accomplish small tasks without the need to delve into Git-Python
In addition it is possible to execute every Git command with Python
syntax, by means of the git.cmd.Git class.
- The ModulesAccess interface has been revised. Basically, all that
applies to the selection of modules, that can be done with modinfo,
has been assembled into one method of ModulesAccess, and is thus
available to other plug-ins.
- Plug-in authors can easily use the new configuration mechanism. The
only task is to instantiate a section object that describes all
options and their default values. After that all active settings can
be retrieved through one call. The commit e1c6809 adds a
configuration section to the status plug-in and is very instructive in
- Importing anything from xjhbuild will install an excepthook that will
display only the message and not the traceback of exceptions with a name
that ends in the term CommonError. Setting the environment variable
XJHBUILD_SHOW_TRACEBACK will show the traceback for these exceptions,
Let me drag you into this:
There are several areas which could need improvement and/or extension.
If you like this project and want to contribute, or if you already have
some useful modifications and/or plug-ins, just contact me directly or
through the trackers on sf.net.
Here are just a few ideas about what could be useful. Of course, other
contribution are welcome, too.
* An installation procedure.
* A plug-in or library module that can start repo-browsers: A simple GUI
window that lists modules (e.g. results from modinfo or status) and
that can open a repository browser (gitk, qgit, giggle, etc) for any
of those modules.
* A patch management plug-in: It could drive git-format-patch
and change the subject to include the module-name and could save patch
series in a sub-folder of a configurable location. It could take a
subject on the command line and use that for the sub-folder and put it
in the cover letter automatically. I guess this could benefit from
further discussion what is needed. For example there could be support
for tags like v2, v7, revoked, etc to facilitate automatic Patchwork
* The plug-in 'spool' could be extended to save/restore repository
states into/from an XML file.
* A plug-in that fetches upstream independent on the current state:
'update' does only fetch changes when on a remote tracking branch and
does that by means of a 'pull --rebase'.
Dirk Wallenstein (153):
config: Update the documentation in the config template
modaccess: Rename method to get_state_preselection
modaccess: Move the dependency sorting function into modaccess
modaccess: Comprehensive module selection with get_git_modules
modinfo: Formulate the sort request variable positively
modaccess: Relocate get_modules_by_name in the same scope
modaccess: Always filter all non-git modules
modaccess: Introduce a select-flags constant to omit filtering
modules: remove the obsolete modules file my.modules
doc: get_git_modules can take any empty sequence of modules
modinfo: Make constants camel-case
modinfo: Limit selection flags alias to scope of use
Write the expand alias message to stderr
modaccess: Improve selection flag comments
modaccess: Add a 'skipped' select flag
modinfo: Fix silent swallowing of skipped modules
Use parentheses to 'from-import' multiple definitions
depcomp: Style changes without behavioral impact
Use the conventional 'Error' suffix for exceptions
Compare the singleton None with 'is' or 'is not'
utils: Introduce the NamedFlags class
utils: Introduce a function to read from a stdin pipe
utils: Introduce the FormattedTerminalOutput class
Add a Git-Python submodule and install it
lib: Introduce the gitaccess library module
plugins: Introduce the 'status' plug-in
Remove the period from every short help
Add short help for the status plug-in
Explicitly derive from 'object'
Remove unused imports
Prompt: Handle Git version without --show-toplevel option
CWD-Module: Handle Git version without --show-toplevel option
Improve the style of import statements
Add a 'Requirements' section to the README file
Add a JHBuild submodule and install it
Add 'uninstall' target to the Makefile
Remove Gnome specific help from tinderbox log
lib: Rename the library module config to conf
Implement 'show-modules-to-skip' as a plug-in
Move all configuration constants into conf.py
Move moduleset section into msspec lib module
Implement moduleset translation errors with exceptions
lib: Introduce the jhbuild_cmdline library module
Generate CWD-Module errors on the call site
Don't validate checkoutroot in get_cwd_module()
Move CWD-Getter into cwdmodule library module
cwdmodule: Simplify adding CWD-Module to JHBuild command line
Sync import statement syntax
Move RGLC section into the library module 'repogroup'
hooks: Use the install-location for hooks directly
hooks: Define the hooks subdir-name in the configuration
hooks: Move Hooks class into hooks library module
Move the PlugIns class into the new library module 'plugins'.
Implement 'get-cwd-module' as a plug-in
Implement 'set-cwd-module' as a plug-in
Prevent whitespace in all parts of the callback
Add envvar XJHBUILD_WANT_CWD_MODULE
Implement 'expand-modules' as a plug-in
Extract repo-group initialization into the lib module 'init'
Rename the concept 'Internal Commands' to 'Built-ins'
Clean up the main script
Simplify command line parsing
Add a patch to change JHBuild configuration defaults
Move JHBuild config parse into jhbuild_config.py
Move FormattedTerminalOutput into lib module 'termout'
Define absolute paths to user config locations
Let repogroup provide paths to locations within the xjh-dir
doc: Add status argument in the tinderwrap help display
Add executable bit to modinfo
modinfo: remove version option
plugins: Remove sort-u
Upgrade subcommand aliases to full command aliases
Install excepthook that omits a traceback for *CommonError
repogroup: Throw *CommonError exception instead of failing
Switch to a new configuration mechanism
status: Split color-config at any non-word char-sequence
status: Add a configuration section
plugins: Ensure config compatible plug-in names
Introduce the config plug-in
init: Allow to create a new repogroup with path spec only
Use *CommonError exceptions instead of error_out()
config: Display string repr of options not python repr
Add reload_config_files() method to XjhConfigParser
Generalize location prefix names
config: Generate an unknown sections warning
repogroup: Add two helper functions
Remove the implicit subcommand
Rename option to want-cwd-module
Add JHBuild buildroot layout patch
Change boolean string representation to yes/no
More reliable list item quotation in string representation
Use the xjh config mechanism for build-configuration
Relocate all main sections below 'core'
Add a moduleset repository configuration section
Add *CommonError base exception to the 'config' plug-in
config: Add options to edit configuration files
init: Make command line parsing consistent
init: Add option to download project configuration
Extract configsection library module from xjhconfig
configsection: Remove xjhbuild.utils dependency
status: Decode author name into unicode
config: Allow to print only non-default values
Remove the plug-in 'show-config'
Install distributed hooks, plug-ins, modulesets into library
Allow to run xjhbuild from the source directory
config: Allow to suppress empty sections
Add a --verbose option to the 'config' plug-in
Overhaul the help output of the 'config' plug-in
Remove checkoutroot arg from get_moduleset_uri()
configsection: Add abstract method get_all_options()
Make configuration available to non-Python plugins
configsection: Rename get_option() to get_value()
Make envvars available as XjhEnvironment properties
Include dependencies by default for modinfo and status
xjh.modules: Make use of a default repository
Generate a CommonError for invalid moduleset specs
Use a singleton meta-class
modules: Add xkbcommon moduleset
xjhconfig: Prevent accessing config from a different repo-group
gitaccess: Add get_commit() to GitRepoInfoAccess
Claim standard input for module selection
status: Use only standard input for module selection
env: Add properties for command line option envvars
Implement callback with main command line option
Deprecate the environment-switch envvar
Make X-JHBuild envvars available in envvar config section
gitaccess: Allow to directly query upstream relation
status: Formulate color toggle config-option positively
msspec: Add a function to determine the moduleset name/id
jhbuild_call: Add lib module to execute JHBuild commands
CommonError: Print only the basename of the script
xjh.modules: Rename pthread-stubs to libpthread-stubs
Persistently switch to a config-token and/or moduleset
Introduce the plug-in 'diff-ms' to compare modulesets
Add three small JHBuild patches
gitaccess: Try to describe detached HEAD with tag
init: Call expanduser() on the destination path argument
Optionally hide version selections from JHBuild modules
Restore configuration reload opportunity at JHBuild prompt
Move Xorg configuration defaults into xjhrc
repogroup: Less errors for the xjh directory structure
init: Add the --restore option
spool: Suppress SHA display by default
spool: Config/Cmdline options for module switch requests
msspec: Automatically clone the moduleset-repository
init: Trigger moduleset-repo clone if necessary
Use the new moduleset repository for Xorg
README: Update for the next release
config: Remove edit-configuration command line options
Fix online-access error generation
More information about the xorg-devel