Manual: Remove tabs, indent consistently
[?]
Feb 28, 2012, 2:38 PM
GYPHTT4MZS75JYMQES2DZPV2D47EJW5HTQWYWDYXE7YJ6FFHPAYACDependencies
- [2]
HN7JDKV3doc: Mention SQLite. - [3]
CPQQFQMZdoc: Reintegrate the intro by Visser & Dolstra from `manual.html'. - [4]
W5GW4E22Don't install hydra-create - [5]
NIUHJZPRremoved some whitespace - [6]
EKNK5AHQdoc: Augment the "Installation" section with material by Visser & Dolstra. - [7]
A6EKITA6Update the docs to reflect the renaming - [8]
3463ZP55doc: Mention the 3 processes that make up Hydra. - [9]
KBW3FDZ2Merge remote branch 'remotes/origin/master'
Change contents
- edit in doc/manual/installation.xml at line 4
<title>Installation</title> - replacement in doc/manual/installation.xml at line 7
<title>Installation</title><para>This chapter explains how to install Hydra on your own build farm server.</para> - edit in doc/manual/installation.xml at line 11
<section><title>Prerequisites</title> - replacement in doc/manual/installation.xml at line 14
This chapter explains how to install Hydra on your own build farm server.</para>To install and use Hydra you need to have installed the following dependencies: - replacement in doc/manual/installation.xml at line 16
<section><title>Prerequisites</title><para>To install and use Hydra you need to have installed the following dependencies:<itemizedlist><listitem>Nix</listitem><listitem>either PostgreSQL or SQLite</listitem><listitem>many Perl packages, notably Catalyst, EmailSender,and NixPerl (see the <linkxlink:href="https://svn.nixos.org/repos/nix/nixpkgs/trunk/pkgs/development/tools/misc/hydra/default.nix">Hydraexpression in Nixpkgs</link> for the completelist).</listitem></itemizedlist> - replacement in doc/manual/installation.xml at line 26[5.106]→[5.564:588](∅→∅),[5.564]→[5.564:588](∅→∅),[5.588]→[5.107:500](∅→∅),[5.500]→[5.697:722](∅→∅),[5.697]→[5.697:722](∅→∅)
<itemizedlist><listitem>Nix</listitem><listitem>either PostgreSQL or SQLite</listitem><listitem>many Perl packages, notably Catalyst,EmailSender, and NixPerl (see the <linkxlink:href="https://svn.nixos.org/repos/nix/nixpkgs/trunk/pkgs/development/tools/misc/hydra/default.nix">Hydraexpression in Nixpkgs</link> for the completelist).</listitem></itemizedlist>At the moment, Hydra runs only on GNU/Linux(<emphasis>i686-linux</emphasis> and<emphasis>x86_64_linux</emphasis>).</para> - replacement in doc/manual/installation.xml at line 31
At the moment, Hydra runs only on GNU/Linux(<emphasis>i686-linux</emphasis> and<emphasis>x86_64_linux</emphasis>).</para><para>For small projects, Hydra can be run on any reasonably modernmachine. For individual projects you can even run Hydra on alaptop. However, the charm of a buildfarm server is usually thatit operates without disturbing the developer's workingenvironment and can serve releases over the internet. Inconjunction you should typically have your source codeadministered in a version management system, such assubversion. Therefore, you will probably want to install aserver that is connected to the internet. To scale up to largeand/or many projects, you will need at least a considerableamount of diskspace to store builds. Since Hydra can schedulemultiple simultaneous build jobs, it can be useful to have amulti-core machine, and/or attach multiple build machines in anetwork to the central Hydra server.</para> - replacement in doc/manual/installation.xml at line 48
<para>For small projects, Hydra can be run on any reasonablymodern machine. For individual projects you can even runHydra on a laptop. However, the charm of a buildfarm serveris usually that it operates without disturbing thedeveloper's working environment and can serve releases overthe internet. In conjunction you should typically have yoursource code administered in a version management system,such as subversion. Therefore, you will probably want toinstall a server that is connected to the internet. To scaleup to large and/or many projects, you will need at least aconsiderable amount of diskspace to store builds. SinceHydra can schedule multiple simultaneous build jobs, it canbe useful to have a multi-core machine, and/or attachmultiple build machines in a network to the central Hydraserver.</para><para>Of course we think it is a good idea to use the <ahref="http://nixos.org/nixos">NixOS</a> GNU/Linuxdistribution for your buildfarm server. But this is not arequirement. The Nix software deployment system can beinstalled on any GNU/Linux distribution in parallel to theregular package management system. Thus, you can use Hydraon a Debian, Fedora, SuSE, or Ubuntu system.</para><para>Of course we think it is a good idea to use the <ahref="http://nixos.org/nixos">NixOS</a> GNU/Linux distributionfor your buildfarm server. But this is not a requirement. TheNix software deployment system can be installed on any GNU/Linuxdistribution in parallel to the regular package managementsystem. Thus, you can use Hydra on a Debian, Fedora, SuSE, orUbuntu system.</para> - replacement in doc/manual/installation.xml at line 58
</section></section> - replacement in doc/manual/installation.xml at line 60
<section><title>Getting Nix</title><section><title>Getting Nix</title> - replacement in doc/manual/installation.xml at line 63
<para>If your server runs NixOS you are all set to continue withinstallation of Hydra. Otherwise you first need to installNix. The latest stable version can be found one <linkxlink:href="http://nixos.org/nix/download.html">the Nix website</link>, along with a manual, which includes installationinstructions.</para></section><para>If your server runs NixOS you are all set to continue withinstallation of Hydra. Otherwise you first need to install Nix.The latest stable version can be found one <linkxlink:href="http://nixos.org/nix/download.html">the Nix website</link>, along with a manual, which includes installationinstructions.</para></section> - replacement in doc/manual/installation.xml at line 73
<section><title>Installation</title><section><title>Installation</title> - replacement in doc/manual/installation.xml at line 76
<para>Hydra can be installed using Nixpkgs:<para>Hydra can be installed using Nixpkgs: - replacement in doc/manual/installation.xml at line 79
<screen>nix-env -Ai hydra -f /path/to/nixpkgs</screen><screen>nix-env -Ai hydra -f /path/to/nixpkgs</screen> - replacement in doc/manual/installation.xml at line 82
This makes the tools available in your Nix user environment,<literal>$HOME/.nix-profile</literal> by default.</para>This makes the tools available in your Nix user environment,<literal>$HOME/.nix-profile</literal> by default.</para> - replacement in doc/manual/installation.xml at line 86[5.2591]→[5.953:968](∅→∅),[5.953]→[5.953:968](∅→∅),[5.968]→[5.2592:2930](∅→∅),[5.2930]→[5.1182:1266](∅→∅),[5.1182]→[5.1182:1266](∅→∅)
<para>Alternatively, the latest development snapshot can beinstalled by visiting the URL<link xlink:href="http://hydra.nixos.org/view/hydra/unstable"><literal>http://hydra.nixos.org/view/hydra/unstable</literal></link>and use the one-click install available at one of the build pages. You can alsoinstall Hydra through the channel by performing the following commands:<para>Alternatively, the latest development snapshot can be installedby visiting the URL <linkxlink:href="http://hydra.nixos.org/view/hydra/unstable"><literal>http://hydra.nixos.org/view/hydra/unstable</literal></link>and use the one-click install available at one of the buildpages. You can also install Hydra through the channel byperforming the following commands: - replacement in doc/manual/installation.xml at line 98
</para></para> - replacement in doc/manual/installation.xml at line 100
<para>Command completion should reveal a number of command-line tools from Hydra:<para>Command completion should reveal a number of command-line tools from Hydra: - replacement in doc/manual/installation.xml at line 103
<screen><screen> - replacement in doc/manual/installation.xml at line 106[4.118]→[5.3255:3264](∅→∅),[5.136]→[5.3255:3264](∅→∅),[5.3255]→[5.3255:3264](∅→∅),[5.3264]→[5.1415:1430](∅→∅),[5.1415]→[5.1415:1430](∅→∅)
</para></section></para></section> - replacement in doc/manual/installation.xml at line 109[5.1431]→[5.1431:1445](∅→∅),[5.1445]→[2.0:45](∅→∅),[2.45]→[5.1486:1501](∅→∅),[5.1486]→[5.1486:1501](∅→∅),[5.1501]→[2.46:232](∅→∅)
<section><title>Creating the database</title><para>Hydra stores its results in a database, which can be aPostgreSQL or SQLite database. The latter is easier tosetup, but the former scales better.</para><section><title>Creating the database</title><para>Hydra stores its results in a database, which can be aPostgreSQL or SQLite database. The latter is easier to setup,but the former scales better.</para><para>To setup a PostgreSQL database with <emphasis>hydra</emphasis>as database name and user name, issue the following commands: - replacement in doc/manual/installation.xml at line 121
<para>To setup a PostgreSQLdatabase with <emphasis>hydra</emphasis> as database nameand user name, issue the following commands:<screen><screen> - replacement in doc/manual/installation.xml at line 126
Note that <emphasis>$prefix</emphasis> is the location ofHydra in the nix store.</para>Note that <emphasis>$prefix</emphasis> is the location of Hydrain the nix store.</para> - replacement in doc/manual/installation.xml at line 131
<para>For SQLite, the following command is all it takes tocreate the database:<para>For SQLite, the following command is all it takes to create thedatabase: - replacement in doc/manual/installation.xml at line 135
<screen>cat $prefix/share/hydra/sql/hydra-sqlite.sql | sqlite3 /path/to/hydra.sqlite</screen></para><screen>cat $prefix/share/hydra/sql/hydra-sqlite.sql | sqlite3 /path/to/hydra.sqlite</screen></para> - replacement in doc/manual/installation.xml at line 139
<para>To add a user <emphasis>root</emphasis> with <emphasis>admin</emphasis> privileges, execute:<screen><para>To add a user <emphasis>root</emphasis> with<emphasis>admin</emphasis> privileges, execute:<screen> - replacement in doc/manual/installation.xml at line 144
echo "INSERT INTO UserRoles(userName, role) values('root', 'admin');" | psql hydra</screen>For SQLite the same commands can be used, with<command>psql hydra</command> replaced by<command>sqlite3 /path/to/hydra.sqlite</command>.</para>echo "INSERT INTO UserRoles(userName, role) values('root', 'admin');" | psql hydra</screen>For SQLite the same commands can be used, with <command>psqlhydra</command> replaced by <command>sqlite3/path/to/hydra.sqlite</command>.</para> - replacement in doc/manual/installation.xml at line 151
<para>Hydra uses an environment variable to know which databaseshould be used, and a variable which point to a locationthat holds some state. To set these variables for aPostgreSQL database, add the following to the<filename>.profile</filename> of the user running theHydra services.<para>Hydra uses an environment variable to know which database shouldbe used, and a variable which point to a location that holdssome state. To set these variables for a PostgreSQL database,add the following to the <filename>.profile</filename> of theuser running the Hydra services. - replacement in doc/manual/installation.xml at line 158
<screen><screen> - replacement in doc/manual/installation.xml at line 162
Make sure that the <emphasis>HYDRA_DATA</emphasis>directory exists and is writable for the user which willrun the Hydra services. For a SQLite database, the<varname>HYDRA_DBI</varname> should be set to somethinglike <literal>dbi:SQLite:/path/to/hydra.sqlite</literal>Make sure that the <emphasis>HYDRA_DATA</emphasis> directoryexists and is writable for the user which will run the Hydraservices. For a SQLite database, the<varname>HYDRA_DBI</varname> should be set to something like<literal>dbi:SQLite:/path/to/hydra.sqlite</literal></para></section> - replacement in doc/manual/installation.xml at line 170
</para></section><section><title>Getting Started</title> - replacement in doc/manual/installation.xml at line 173[5.3012]→[5.3012:3026](∅→∅),[5.3026]→[5.0:40](∅→∅),[5.40]→[5.3065:3080](∅→∅),[5.3065]→[5.3065:3080](∅→∅),[5.3080]→[5.41:105](∅→∅),[5.105]→[5.3143:3164](∅→∅),[5.3143]→[5.3143:3164](∅→∅)
<section><title>Getting Started</title><para>To start the Hydra web server, execute:<screen><para>To start the Hydra web server, execute:<screen> - replacement in doc/manual/installation.xml at line 178[5.3190]→[5.106:268](∅→∅),[5.268]→[5.3328:3344](∅→∅),[5.3328]→[5.3328:3344](∅→∅),[5.3344]→[5.269:278](∅→∅),[5.278]→[5.160:220](∅→∅),[5.220]→[5.341:411](∅→∅),[5.341]→[5.341:411](∅→∅)
When the server is started, you can browse to<ulink>http://localhost:3000/</ulink> to start configuringyour Hydra instance.</para><para>The <command>hydra-server</command> command launches theweb server. There are two other processes that come intoplay:When the server is started, you can browse to<ulink>http://localhost:3000/</ulink> to start configuringyour Hydra instance.</para> - replacement in doc/manual/installation.xml at line 183[5.3360]→[5.412:729](∅→∅),[5.729]→[5.221:272](∅→∅),[5.272]→[5.783:1020](∅→∅),[5.783]→[5.783:1020](∅→∅),[5.1020]→[5.273:327](∅→∅),[5.327]→[5.1077:1113](∅→∅),[5.1077]→[5.1077:1113](∅→∅)
<itemizedlist><listitem>The <emphasis>evaluator</emphasis> is responsible forperidically evaluating job sets, checking out theirdependencies off their version control systems (VCS),and queueing new builds if the result of the evaluationchanged. It is launched by the<command>hydra-evaluator</command> command.</listitem><listitem>The <emphasis>queue runner</emphasis> launches builds(using Nix) as they are queued by the evaluator,scheduling them onto the configured Nix hosts. It islaunched using the<command>hydra-queue-runner</command> command.</listitem></itemizedlist><para>The <command>hydra-server</command> command launches the webserver. There are two other processes that come into play: - replacement in doc/manual/installation.xml at line 187
All three processes must be running for Hydra to be fullyfunctional, though it's possible to temporarily stop any oneof them for maintenance purposes, for instance.</para></section><itemizedlist><listitem>The <emphasis>evaluator</emphasis> is responsible forperidically evaluating job sets, checking out theirdependencies off their version control systems (VCS), andqueueing new builds if the result of the evaluation changed.It is launched by the <command>hydra-evaluator</command>command.</listitem><listitem>The <emphasis>queue runner</emphasis> launches builds (usingNix) as they are queued by the evaluator, scheduling themonto the configured Nix hosts. It is launched using the<command>hydra-queue-runner</command> command.</listitem></itemizedlist> - edit in doc/manual/installation.xml at line 204
All three processes must be running for Hydra to be fullyfunctional, though it's possible to temporarily stop any one ofthem for maintenance purposes, for instance.</para></section> - edit in doc/manual/installation.xml at line 212[5.3753]
<!--Local Variables:indent-tabs-mode: nilispell-local-dictionary: "american"End:--> - replacement in doc/manual/introduction.xml at line 22
<listitem> <emphasis>Portability testing</emphasis>: Thesoftware may need to be built and tested on many differentplatforms. It is infeasible for each developer to do thisbefore every commit.</listitem><listitem> <emphasis>Portability testing</emphasis>: Thesoftware may need to be built and tested on many differentplatforms. It is infeasible for each developer to do thisbefore every commit.</listitem> - replacement in doc/manual/introduction.xml at line 28
<listitem> Likewise, many projects have very large test sets(e.g., regression tests in a compiler, or stress tests in aDBMS) that can take hours or days to run to completion.</listitem><listitem> Likewise, many projects have very large test sets(e.g., regression tests in a compiler, or stress tests in aDBMS) that can take hours or days to run to completion.</listitem> - replacement in doc/manual/introduction.xml at line 33
<listitem> Many kinds of static and dynamic analyses can beperformed as part of the tests, such as code coverage runs andstatic analyses.</listitem><listitem> Many kinds of static and dynamic analyses can beperformed as part of the tests, such as code coverage runs andstatic analyses.</listitem> - replacement in doc/manual/introduction.xml at line 38
<listitem> It may also be necessary to build many different<emphasis>variants</emphasis> of the software. For instance,it may be necessary to verify that the component builds withvarious versions of a compiler.</listitem><listitem> It may also be necessary to build many different<emphasis>variants</emphasis> of the software. For instance,it may be necessary to verify that the component builds withvarious versions of a compiler.</listitem> - replacement in doc/manual/introduction.xml at line 44
<listitem> Developers typically use incremental building totest their changes (since a full build may take too long), butthis is unreliable with many build management tools (such asMake), i.e., the result of the incremental build might differfrom a full build.</listitem><listitem> Developers typically use incremental building totest their changes (since a full build may take too long), butthis is unreliable with many build management tools (such asMake), i.e., the result of the incremental build might differfrom a full build.</listitem> - replacement in doc/manual/introduction.xml at line 51
<listitem> It ensures that the software can be built from thesources under revision control. Users of version managementsystems such as CVS and Subversion often forget to placesource files under revision control.</listitem><listitem> It ensures that the software can be built from thesources under revision control. Users of version managementsystems such as CVS and Subversion often forget to placesource files under revision control.</listitem> - replacement in doc/manual/introduction.xml at line 57
<listitem> The machines on which the continuous integrationsystem runs ideally provides a clean, well-defined buildenvironment. If this environment is administered throughproper SCM techniques, then builds produced by the system canbe reproduced. In contrast, developer work environments aretypically not under any kind of SCM control.</listitem><listitem> The machines on which the continuous integrationsystem runs ideally provides a clean, well-defined buildenvironment. If this environment is administered throughproper SCM techniques, then builds produced by the system canbe reproduced. In contrast, developer work environments aretypically not under any kind of SCM control.</listitem> - replacement in doc/manual/introduction.xml at line 65
<listitem> In large projects, developers often work on aparticular component of the project, and do not build and testthe composition of those components (again since this islikely to take too long). To prevent the phenomenon of ``bigbang integration'', where components are only tested togethernear the end of the development process, it is important totest components together as soon as possible (hence<emphasis>continuous integration</emphasis>).</listitem><listitem> In large projects, developers often work on aparticular component of the project, and do not build and testthe composition of those components (again since this islikely to take too long). To prevent the phenomenon of ``bigbang integration'', where components are only tested togethernear the end of the development process, it is important totest components together as soon as possible (hence<emphasis>continuous integration</emphasis>).</listitem> - replacement in doc/manual/introduction.xml at line 75
<listitem> It allows software to be<emphasis>released</emphasis> by automatically creatingpackages that users can download and install. To do thismanually represents an often prohibitive amount of work, asone may want to produce releases for many different platforms:e.g., installers for Windows and Mac OS X, RPM or Debianpackages for certain Linux distributions, and so on.</listitem><listitem> It allows software to be<emphasis>released</emphasis> by automatically creatingpackages that users can download and install. To do thismanually represents an often prohibitive amount of work, asone may want to produce releases for many different platforms:e.g., installers for Windows and Mac OS X, RPM or Debianpackages for certain Linux distributions, and so on.</listitem> - replacement in doc/manual/introduction.xml at line 95
<listitem>It obtains the latest version of the component'ssource code from the version management system.</listitem><listitem>It obtains the latest version of the component'ssource code from the version management system.</listitem> - replacement in doc/manual/introduction.xml at line 99
<listitem> It runs the component's build process (whichpresumably includes the execution of the component's testset).</listitem><listitem> It runs the component's build process (whichpresumably includes the execution of the component's testset).</listitem> - replacement in doc/manual/introduction.xml at line 104
<listitem> It presents the results of the build (such as errorlogs and releases) to the developers, e.g., by producing a webpage.</listitem><listitem> It presents the results of the build (such as errorlogs and releases) to the developers, e.g., by producing a webpage.</listitem> - replacement in doc/manual/introduction.xml at line 117
<listitem> They do not manage the <emphasis>buildenvironment</emphasis>. The build environment consists of thedependencies necessary to perform a build action, e.g.,compilers, libraries, etc. Setting up the environment istypically done manually, and without proper SCM control (so itmay be hard to reproduce a build at a later time). Manualmanagement of the environment scales poorly in the number ofconfigurations that must be supported. For instance, supposethat we want to build a component that requires a certaincompiler X. We then have to go to each machine and install X.If we later need a newer version of X, the process must berepeated all over again. An ever worse problem occurs ifthere are conflicting, mutually exclusive versions of thedependencies. Thus, simply installing the latest version isnot an option. Of course, we can install these components indifferent directories and manually pass the appropriate pathsto the build processes of the various components. But this isa rather tiresome and error-prone process.</listitem><listitem> They do not manage the <emphasis>buildenvironment</emphasis>. The build environment consists of thedependencies necessary to perform a build action, e.g.,compilers, libraries, etc. Setting up the environment istypically done manually, and without proper SCM control (so itmay be hard to reproduce a build at a later time). Manualmanagement of the environment scales poorly in the number ofconfigurations that must be supported. For instance, supposethat we want to build a component that requires a certaincompiler X. We then have to go to each machine and install X.If we later need a newer version of X, the process must berepeated all over again. An ever worse problem occurs ifthere are conflicting, mutually exclusive versions of thedependencies. Thus, simply installing the latest version isnot an option. Of course, we can install these components indifferent directories and manually pass the appropriate pathsto the build processes of the various components. But this isa rather tiresome and error-prone process.</listitem> - replacement in doc/manual/introduction.xml at line 137
<listitem> They do not easily support <emphasis>variability in softwaresystems</emphasis>. A system may have a great deal of build-timevariability: optional functionality, whether to build a debug orproduction version, different versions of dependencies, and so on.(For instance, the Linux kernel now has over 2,600 build-timeconfiguration switches.) It is therefore important that a continuousintegration tool can easily select and test different instances fromthe configuration space of the system to reveal problems, such aserroneous interactions between features. In a continuous integrationsetting, it is also useful to test different combinations of versionsof subsystems, e.g., the head revision of a component against stablereleases of its dependencies, and vice versa, as this can revealvarious integration problems.</listitem><listitem> They do not easily support <emphasis>variability in softwaresystems</emphasis>. A system may have a great deal of build-timevariability: optional functionality, whether to build a debug orproduction version, different versions of dependencies, and so on.(For instance, the Linux kernel now has over 2,600 build-timeconfiguration switches.) It is therefore important that a continuousintegration tool can easily select and test different instances fromthe configuration space of the system to reveal problems, such aserroneous interactions between features. In a continuous integrationsetting, it is also useful to test different combinations of versionsof subsystems, e.g., the head revision of a component against stablereleases of its dependencies, and vice versa, as this can revealvarious integration problems.</listitem> - edit in doc/manual/introduction.xml at line 261[5.3974]
<!--Local Variables:indent-tabs-mode: nilispell-local-dictionary: "american"End:-->