<title>Installation</title>

    <title>Installation</title>

  <para>
    This chapter explains how to install Hydra on your own build farm server.
  </para>

  <section>
    <title>Prerequisites</title>

      This chapter explains how to install Hydra on your own build farm server.
    </para>

      To install and use Hydra you need to have installed the following dependencies:

    <section>
        <title>Prerequisites</title>
        <para>
	        To install and use Hydra you need to have installed the following dependencies:

      <itemizedlist>
        <listitem>Nix</listitem>
        <listitem>either PostgreSQL or SQLite</listitem>
        <listitem>many Perl packages, notably Catalyst, EmailSender,
        and NixPerl (see the <link
        xlink:href="https://svn.nixos.org/repos/nix/nixpkgs/trunk/pkgs/development/tools/misc/hydra/default.nix">Hydra
        expression in Nixpkgs</link> for the complete
        list).</listitem>
      </itemizedlist>

	        <itemizedlist>
	            <listitem>Nix</listitem>
	            <listitem>either PostgreSQL or SQLite</listitem>
		    <listitem>many Perl packages, notably Catalyst,
		    EmailSender, and NixPerl (see the <link
		    xlink:href="https://svn.nixos.org/repos/nix/nixpkgs/trunk/pkgs/development/tools/misc/hydra/default.nix">Hydra
		    expression in Nixpkgs</link> for the complete
		    list).</listitem>
	        </itemizedlist>

      At the moment, Hydra runs only on GNU/Linux
      (<emphasis>i686-linux</emphasis> and
      <emphasis>x86_64_linux</emphasis>).
    </para>

	        At the moment, Hydra runs only on GNU/Linux
	        (<emphasis>i686-linux</emphasis> and
	        <emphasis>x86_64_linux</emphasis>).
        </para>

    <para>
      For small projects, Hydra can be run on any reasonably modern
      machine. For individual projects you can even run Hydra on a
      laptop. However, the charm of a buildfarm server is usually that
      it operates without disturbing the developer's working
      environment and can serve releases over the internet. In
      conjunction you should typically have your source code
      administered in a version management system, such as
      subversion. Therefore, you will probably want to install a
      server that is connected to the internet. To scale up to large
      and/or many projects, you will need at least a considerable
      amount of diskspace to store builds. Since Hydra can schedule
      multiple simultaneous build jobs, it can be useful to have a
      multi-core machine, and/or attach multiple build machines in a
      network to the central Hydra server.
    </para>

	<para>
	  For small projects, Hydra can be run on any reasonably
	  modern machine. For individual projects you can even run
	  Hydra on a laptop. However, the charm of a buildfarm server
	  is usually that it operates without disturbing the
	  developer's working environment and can serve releases over
	  the internet. In conjunction you should typically have your
	  source code administered in a version management system,
	  such as subversion. Therefore, you will probably want to
	  install a server that is connected to the internet. To scale
	  up to large and/or many projects, you will need at least a
	  considerable amount of diskspace to store builds. Since
	  Hydra can schedule multiple simultaneous build jobs, it can
	  be useful to have a multi-core machine, and/or attach
	  multiple build machines in a network to the central Hydra
	  server.
	</para>
	<para>
	  Of course we think it is a good idea to use the <a
	  href="http://nixos.org/nixos">NixOS</a> GNU/Linux
	  distribution for your buildfarm server. But this is not a
	  requirement.  The Nix software deployment system can be
	  installed on any GNU/Linux distribution in parallel to the
	  regular package management system. Thus, you can use Hydra
	  on a Debian, Fedora, SuSE, or Ubuntu system.
	</para>

    <para>
      Of course we think it is a good idea to use the <a
      href="http://nixos.org/nixos">NixOS</a> GNU/Linux distribution
      for your buildfarm server. But this is not a requirement.  The
      Nix software deployment system can be installed on any GNU/Linux
      distribution in parallel to the regular package management
      system. Thus, you can use Hydra on a Debian, Fedora, SuSE, or
      Ubuntu system.
    </para>

    </section>

  </section>

    <section>
      <title>Getting Nix</title>

  <section>
    <title>Getting Nix</title>

      <para>
	If your server runs NixOS you are all set to continue with
	installation of Hydra. Otherwise you first need to install
	Nix.  The latest stable version can be found one <link
	xlink:href="http://nixos.org/nix/download.html">the Nix web
	site</link>, along with a manual, which includes installation
	instructions.
      </para>
    </section>

    <para>
      If your server runs NixOS you are all set to continue with
      installation of Hydra. Otherwise you first need to install Nix.
      The latest stable version can be found one <link
      xlink:href="http://nixos.org/nix/download.html">the Nix web
      site</link>, along with a manual, which includes installation
      instructions.
    </para>
  </section>

    <section>
        <title>Installation</title>

  <section>
    <title>Installation</title>

	<para>
	  Hydra can be installed using Nixpkgs:

    <para>
      Hydra can be installed using Nixpkgs:

	  <screen>
	    nix-env -Ai hydra -f /path/to/nixpkgs</screen>

      <screen>
nix-env -Ai hydra -f /path/to/nixpkgs</screen>

	  This makes the tools available in your Nix user environment,
	  <literal>$HOME/.nix-profile</literal> by default.
	</para>

      This makes the tools available in your Nix user environment,
      <literal>$HOME/.nix-profile</literal> by default.
    </para>

        <para>
            Alternatively, the latest development snapshot can be
            installed by visiting the URL
	    <link xlink:href="http://hydra.nixos.org/view/hydra/unstable"><literal>http://hydra.nixos.org/view/hydra/unstable</literal></link>
            and use the one-click install available at one of the build pages. You can also  
            install Hydra through the channel by performing the following commands:

    <para>
      Alternatively, the latest development snapshot can be installed
      by visiting the URL <link
      xlink:href="http://hydra.nixos.org/view/hydra/unstable"><literal>http://hydra.nixos.org/view/hydra/unstable</literal></link>
      and use the one-click install available at one of the build
      pages. You can also install Hydra through the channel by
      performing the following commands:

        </para>

    </para>

	<para>
	  Command completion should reveal a number of command-line tools from Hydra:

    <para>
      Command completion should reveal a number of command-line tools from Hydra:

	  <screen>

          <screen>

	</para>
    </section>

    </para>
  </section>

    <section>
        <title>Creating the database</title>
        <para>
            Hydra stores its results in a database, which can be a
            PostgreSQL or SQLite database.  The latter is easier to
	    setup, but the former scales better.
	</para>

  <section>
    <title>Creating the database</title>
    <para>
      Hydra stores its results in a database, which can be a
      PostgreSQL or SQLite database.  The latter is easier to setup,
      but the former scales better.
    </para>
    <para>
      To setup a PostgreSQL database with <emphasis>hydra</emphasis>
      as database name and user name, issue the following commands:

	<para>To setup a PostgreSQL
            database with <emphasis>hydra</emphasis> as database name
            and user name, issue the following commands:
            <screen>

      <screen>

            Note that <emphasis>$prefix</emphasis> is the location of
	    Hydra in the nix store.
	</para>

      Note that <emphasis>$prefix</emphasis> is the location of Hydra
      in the nix store.
    </para>

	<para>
	    For SQLite, the following command is all it takes to
	    create the database:

    <para>
      For SQLite, the following command is all it takes to create the
      database:

	    <screen>
	      cat $prefix/share/hydra/sql/hydra-sqlite.sql | sqlite3 /path/to/hydra.sqlite
	    </screen>
        </para>

      <screen>
cat $prefix/share/hydra/sql/hydra-sqlite.sql | sqlite3 /path/to/hydra.sqlite</screen>
    </para>

        <para>
            To add a user <emphasis>root</emphasis> with <emphasis>admin</emphasis> privileges, execute:            
            <screen>

    <para>
      To add a user <emphasis>root</emphasis> with
      <emphasis>admin</emphasis> privileges, execute:
      <screen>

echo "INSERT INTO UserRoles(userName, role) values('root', 'admin');" | psql hydra
	    </screen>
	    For SQLite the same commands can be used, with
	    <command>psql hydra</command> replaced by
	    <command>sqlite3 /path/to/hydra.sqlite</command>.
	</para>

echo "INSERT INTO UserRoles(userName, role) values('root', 'admin');" | psql hydra</screen>
      For SQLite the same commands can be used, with <command>psql
      hydra</command> replaced by <command>sqlite3
      /path/to/hydra.sqlite</command>.
    </para>

	<para>
            Hydra uses an environment variable to know which database
            should be used, and a variable which point to a location
            that holds some state. To set these variables for a
            PostgreSQL database, add the following to the
            <filename>.profile</filename> of the user running the
            Hydra services.

    <para>
      Hydra uses an environment variable to know which database should
      be used, and a variable which point to a location that holds
      some state. To set these variables for a PostgreSQL database,
      add the following to the <filename>.profile</filename> of the
      user running the Hydra services.

            <screen>

      <screen>

            Make sure that the <emphasis>HYDRA_DATA</emphasis>
            directory exists and is writable for the user which will
            run the Hydra services.  For a SQLite database, the
            <varname>HYDRA_DBI</varname> should be set to something
            like <literal>dbi:SQLite:/path/to/hydra.sqlite</literal>

      Make sure that the <emphasis>HYDRA_DATA</emphasis> directory
      exists and is writable for the user which will run the Hydra
      services.  For a SQLite database, the
      <varname>HYDRA_DBI</varname> should be set to something like
      <literal>dbi:SQLite:/path/to/hydra.sqlite</literal>
    </para>
  </section>

        </para>
    </section>

  <section>
    <title>Getting Started</title>

    <section>
        <title>Getting Started</title>
        <para>
            To start the Hydra web server, execute:            
            <screen>

    <para>
      To start the Hydra web server, execute:            
      <screen>

            When the server is started, you can browse to
            <ulink>http://localhost:3000/</ulink> to start configuring
            your Hydra instance.
        </para>
	<para>
	  The <command>hydra-server</command> command launches the
	  web server.  There are two other processes that come into
	  play:

      When the server is started, you can browse to
      <ulink>http://localhost:3000/</ulink> to start configuring
      your Hydra instance.
    </para>

	  <itemizedlist>
	    <listitem>
	      The <emphasis>evaluator</emphasis> is responsible for
	      peridically evaluating job sets, checking out their
	      dependencies off their version control systems (VCS),
	      and queueing new builds if the result of the evaluation
	      changed.  It is launched by the
	      <command>hydra-evaluator</command> command.
	    </listitem>
	    <listitem>
	      The <emphasis>queue runner</emphasis> launches builds
	      (using Nix) as they are queued by the evaluator,
	      scheduling them onto the configured Nix hosts.  It is
	      launched using the
	      <command>hydra-queue-runner</command> command.
	    </listitem>
	  </itemizedlist>

    <para>
      The <command>hydra-server</command> command launches the web
      server.  There are two other processes that come into play:

	  All three processes must be running for Hydra to be fully
	  functional, though it's possible to temporarily stop any one
	  of them for maintenance purposes, for instance.
	</para>
	  
    </section>

      <itemizedlist>
        <listitem>
          The <emphasis>evaluator</emphasis> is responsible for
          peridically evaluating job sets, checking out their
          dependencies off their version control systems (VCS), and
          queueing new builds if the result of the evaluation changed.
          It is launched by the <command>hydra-evaluator</command>
          command.
        </listitem>
        <listitem>
          The <emphasis>queue runner</emphasis> launches builds (using
          Nix) as they are queued by the evaluator, scheduling them
          onto the configured Nix hosts.  It is launched using the
          <command>hydra-queue-runner</command> command.
        </listitem>
      </itemizedlist>

      All three processes must be running for Hydra to be fully
      functional, though it's possible to temporarily stop any one of
      them for maintenance purposes, for instance.
    </para>
          
  </section>

<!--
 Local Variables:
 indent-tabs-mode: nil
 ispell-local-dictionary: "american"
 End:
 -->

	<listitem> <emphasis>Portability testing</emphasis>: The
	software may need to be built and tested on many different
	platforms.  It is infeasible for each developer to do this
	before every commit.
	</listitem>

        <listitem> <emphasis>Portability testing</emphasis>: The
        software may need to be built and tested on many different
        platforms.  It is infeasible for each developer to do this
        before every commit.
        </listitem>

	<listitem> Likewise, many projects have very large test sets
	(e.g., regression tests in a compiler, or stress tests in a
	DBMS) that can take hours or days to run to completion.
	</listitem>

        <listitem> Likewise, many projects have very large test sets
        (e.g., regression tests in a compiler, or stress tests in a
        DBMS) that can take hours or days to run to completion.
        </listitem>

	<listitem> Many kinds of static and dynamic analyses can be
	performed as part of the tests, such as code coverage runs and
	static analyses.
	</listitem>

        <listitem> Many kinds of static and dynamic analyses can be
        performed as part of the tests, such as code coverage runs and
        static analyses.
        </listitem>

	<listitem> It may also be necessary to build many different
	<emphasis>variants</emphasis> of the software.  For instance,
	it may be necessary to verify that the component builds with
	various versions of a compiler.
	</listitem>

        <listitem> It may also be necessary to build many different
        <emphasis>variants</emphasis> of the software.  For instance,
        it may be necessary to verify that the component builds with
        various versions of a compiler.
        </listitem>

	<listitem> Developers typically use incremental building to
	test their changes (since a full build may take too long), but
	this is unreliable with many build management tools (such as
	Make), i.e., the result of the incremental build might differ
	from a full build.
	</listitem>

        <listitem> Developers typically use incremental building to
        test their changes (since a full build may take too long), but
        this is unreliable with many build management tools (such as
        Make), i.e., the result of the incremental build might differ
        from a full build.
        </listitem>

	<listitem> It ensures that the  software can be built from the
	sources under  revision control.  Users  of version management
	systems  such as  CVS  and Subversion  often  forget to  place
	source files under revision control.
	</listitem>

        <listitem> It ensures that the  software can be built from the
        sources under  revision control.  Users  of version management
        systems  such as  CVS  and Subversion  often  forget to  place
        source files under revision control.
        </listitem>

	<listitem> The machines on which the continuous integration
	system runs ideally provides a clean, well-defined build
	environment.  If this environment is administered through
	proper SCM techniques, then builds produced by the system can
	be reproduced.  In contrast, developer work environments are
	typically not under any kind of SCM control.
	</listitem>

        <listitem> The machines on which the continuous integration
        system runs ideally provides a clean, well-defined build
        environment.  If this environment is administered through
        proper SCM techniques, then builds produced by the system can
        be reproduced.  In contrast, developer work environments are
        typically not under any kind of SCM control.
        </listitem>

	<listitem> In large projects, developers often work on a
	particular component of the project, and do not build and test
	the composition of those components (again since this is
	likely to take too long).  To prevent the phenomenon of ``big
	bang integration'', where components are only tested together
	near the end of the development process, it is important to
	test components together as soon as possible (hence
	<emphasis>continuous integration</emphasis>).
	</listitem>

        <listitem> In large projects, developers often work on a
        particular component of the project, and do not build and test
        the composition of those components (again since this is
        likely to take too long).  To prevent the phenomenon of ``big
        bang integration'', where components are only tested together
        near the end of the development process, it is important to
        test components together as soon as possible (hence
        <emphasis>continuous integration</emphasis>).
        </listitem>

	<listitem> It allows software to be
	<emphasis>released</emphasis> by automatically creating
	packages that users can download and install.  To do this
	manually represents an often prohibitive amount of work, as
	one may want to produce releases for many different platforms:
	e.g., installers for Windows and Mac OS X, RPM or Debian
	packages for certain Linux distributions, and so on.
	</listitem>

        <listitem> It allows software to be
        <emphasis>released</emphasis> by automatically creating
        packages that users can download and install.  To do this
        manually represents an often prohibitive amount of work, as
        one may want to produce releases for many different platforms:
        e.g., installers for Windows and Mac OS X, RPM or Debian
        packages for certain Linux distributions, and so on.
        </listitem>

	<listitem>It obtains the latest version of the component's
	source code from the version management system.
	</listitem>

        <listitem>It obtains the latest version of the component's
        source code from the version management system.
        </listitem>

	<listitem> It runs the component's build process (which
	presumably includes the execution of the component's test
	set).
	</listitem>

        <listitem> It runs the component's build process (which
        presumably includes the execution of the component's test
        set).
        </listitem>

	<listitem> It presents the results of the build (such as error
	logs and releases) to the developers, e.g., by producing a web
	page.
	</listitem>

        <listitem> It presents the results of the build (such as error
        logs and releases) to the developers, e.g., by producing a web
        page.
        </listitem>

	<listitem> They do not manage the <emphasis>build
	environment</emphasis>.  The build environment consists of the
	dependencies necessary to perform a build action, e.g.,
	compilers, libraries, etc.  Setting up the environment is
	typically done manually, and without proper SCM control (so it
	may be hard to reproduce a build at a later time).  Manual
	management of the environment scales poorly in the number of
	configurations that must be supported.  For instance, suppose
	that we want to build a component that requires a certain
	compiler X.  We then have to go to each machine and install X.
	If we later need a newer version of X, the process must be
	repeated all over again.  An ever worse problem occurs if
	there are conflicting, mutually exclusive versions of the
	dependencies.  Thus, simply installing the latest version is
	not an option.  Of course, we can install these components in
	different directories and manually pass the appropriate paths
	to the build processes of the various components.  But this is
	a rather tiresome and error-prone process.
	</listitem>

        <listitem> They do not manage the <emphasis>build
        environment</emphasis>.  The build environment consists of the
        dependencies necessary to perform a build action, e.g.,
        compilers, libraries, etc.  Setting up the environment is
        typically done manually, and without proper SCM control (so it
        may be hard to reproduce a build at a later time).  Manual
        management of the environment scales poorly in the number of
        configurations that must be supported.  For instance, suppose
        that we want to build a component that requires a certain
        compiler X.  We then have to go to each machine and install X.
        If we later need a newer version of X, the process must be
        repeated all over again.  An ever worse problem occurs if
        there are conflicting, mutually exclusive versions of the
        dependencies.  Thus, simply installing the latest version is
        not an option.  Of course, we can install these components in
        different directories and manually pass the appropriate paths
        to the build processes of the various components.  But this is
        a rather tiresome and error-prone process.
        </listitem>

	<listitem> They do not easily support <emphasis>variability in software
	systems</emphasis>.  A system may have a great deal of build-time
	variability: optional functionality, whether to build a debug or
	production version, different versions of dependencies, and so on.
	(For instance, the Linux kernel now has over 2,600 build-time
	configuration switches.)  It is therefore important that a continuous
	integration tool can easily select and test different instances from
	the configuration space of the system to reveal problems, such as
	erroneous interactions between features.  In a continuous integration
	setting, it is also useful to test different combinations of versions
	of subsystems, e.g., the head revision of a component against stable
	releases of its dependencies, and vice versa, as this can reveal
	various integration problems.
	</listitem>

        <listitem> They do not easily support <emphasis>variability in software
        systems</emphasis>.  A system may have a great deal of build-time
        variability: optional functionality, whether to build a debug or
        production version, different versions of dependencies, and so on.
        (For instance, the Linux kernel now has over 2,600 build-time
        configuration switches.)  It is therefore important that a continuous
        integration tool can easily select and test different instances from
        the configuration space of the system to reveal problems, such as
        erroneous interactions between features.  In a continuous integration
        setting, it is also useful to test different combinations of versions
        of subsystems, e.g., the head revision of a component against stable
        releases of its dependencies, and vice versa, as this can reveal
        various integration problems.
        </listitem>

<!--
 Local Variables:
 indent-tabs-mode: nil
 ispell-local-dictionary: "american"
 End:
 -->