grahamc/hydra - Change GLIPWYWLW6JNCIPMAXPKU64RHO2WDIFYZ5ZNGMYUJKX5Y7ZPSVJQC

Move evaluation errors from evaluations to EvaluationErrors, a new table

DBIx likes to eagerly select all columns without a way to really tell it so. Therefore, this splits this one large column in to its own table.

I'd also like to make "jobsets" use this table too, but that is on hold to stop the bleeding caused by the extreme amount of traffic this is causing.

Created by Graham Christensen on February 2, 2021

GLIPWYWLW6JNCIPMAXPKU64RHO2WDIFYZ5ZNGMYUJKX5Y7ZPSVJQC

Dependencies

In channels

main

upstream

Change contents

Replacement in src/lib/Hydra/Helper/Nix.pm at line 214 [8.436]

B:BD[6.1010] → [7.1274:1344]

∅:D[7.1344] → [6.1084:1085]

B:BD[6.1084] → [6.1084:1085]

        { order_by => "id DESC", rows => $rows, offset => $offset });

[6.1010]

[6.1085]

        { order_by => "me.id DESC", rows => $rows, offset => $offset
        , prefetch => { evaluationerror => [  ] } });

File addition: EvaluationErrors.pm (----------)

[9.477]

use utf8;
package Hydra::Schema::EvaluationErrors;
# Created by DBIx::Class::Schema::Loader
# DO NOT MODIFY THE FIRST PART OF THIS FILE
=head1 NAME
Hydra::Schema::EvaluationErrors
=cut
use strict;
use warnings;
use base 'DBIx::Class::Core';
=head1 COMPONENTS LOADED
=over 4
=item * L<Hydra::Component::ToJSON>
=back
=cut
__PACKAGE__->load_components("+Hydra::Component::ToJSON");
=head1 TABLE: C<evaluationerrors>
=cut
__PACKAGE__->table("evaluationerrors");
=head1 ACCESSORS
=head2 id
  data_type: 'integer'
  is_auto_increment: 1
  is_nullable: 0
  sequence: 'evaluationerrors_id_seq'
=head2 errormsg
  data_type: 'text'
  is_nullable: 1
=head2 errortime
  data_type: 'integer'
  is_nullable: 1
=cut
__PACKAGE__->add_columns(
  "id",
  {
    data_type         => "integer",
    is_auto_increment => 1,
    is_nullable       => 0,
    sequence          => "evaluationerrors_id_seq",
  },
  "errormsg",
  { data_type => "text", is_nullable => 1 },
  "errortime",
  { data_type => "integer", is_nullable => 1 },
);
=head1 PRIMARY KEY
=over 4
=item * L</id>
=back
=cut
__PACKAGE__->set_primary_key("id");
=head1 RELATIONS
=head2 jobsetevals
Type: has_many
Related object: L<Hydra::Schema::JobsetEvals>
=cut
__PACKAGE__->has_many(
  "jobsetevals",
  "Hydra::Schema::JobsetEvals",
  { "foreign.evaluationerror_id" => "self.id" },
  undef,
);
# Created by DBIx::Class::Schema::Loader v0.07049 @ 2021-02-01 20:17:39
# DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:sZIg35KWCO8MOsQ5cfN1IA
__PACKAGE__->add_column(
    "+id" => { retrieve_on_insert => 1 }
);
1;

Deletion in src/lib/Hydra/Schema/JobsetEvals.pm at line 50 [11.65]
B:BD[10.16175] → [10.16175:16176]
B:BD[12.1] → [12.1:55]
```
=head2 errormsg
  data_type: 'text'
  is_nullable: 1
```
Replacement in src/lib/Hydra/Schema/JobsetEvals.pm at line 51 [11.65]
B:BD[12.56] → [12.56:73]
```
=head2 errortime
```
[12.56]
[12.73]
```
=head2 evaluationerror_id
```
Insertion in src/lib/Hydra/Schema/JobsetEvals.pm at line 54 [11.65]
[12.97]
[12.97]
```
  is_foreign_key: 1
```

Replacement in src/lib/Hydra/Schema/JobsetEvals.pm at line 119 [11.65]

∅:D[5.126] → [12.115:237]

B:BD[13.17821] → [12.115:237]

  "errormsg",
  { data_type => "text", is_nullable => 1 },
  "errortime",
  { data_type => "integer", is_nullable => 1 },

[5.126]

[14.3788]

  "evaluationerror_id",
  { data_type => "integer", is_foreign_key => 1, is_nullable => 1 },

Insertion in src/lib/Hydra/Schema/JobsetEvals.pm at line 156 [11.65]

[13.18170]

[10.16664]


=head2 evaluationerror
Type: belongs_to
Related object: L<Hydra::Schema::EvaluationErrors>

Insertion in src/lib/Hydra/Schema/JobsetEvals.pm at line 163 [11.65]

[10.16665]

=cut
__PACKAGE__->belongs_to(
  "evaluationerror",
  "Hydra::Schema::EvaluationErrors",
  { id => "evaluationerror_id" },
  {
    is_deferrable => 0,
    join_type     => "LEFT",
    on_delete     => "SET NULL",
    on_update     => "NO ACTION",
  },
);

Replacement in src/lib/Hydra/Schema/JobsetEvals.pm at line 223 [11.65]

B:BD[11.1130] → [5.229:371]

# Created by DBIx::Class::Schema::Loader v0.07049 @ 2021-01-25 14:44:07
# DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:OVxeYH+eoZZrAsAJ2/mAAA

[11.1130]

[13.18211]

# Created by DBIx::Class::Schema::Loader v0.07049 @ 2021-02-01 20:17:39
# DO NOT MODIFY THIS OR ANYTHING ABOVE! md5sum:SGtK0PwRkbxiMuitQvs4wQ

Replacement in src/root/common.tt at line 479 [16.2803]

B:BD[15.10521] → [4.0:35]

            [% IF eval.errormsg %]

[15.10521]

[4.35]

            [% IF eval.evaluationerror.errormsg %]

Replacement in src/root/jobset-eval.tt at line 92 [17.4844]
B:BD[4.126] → [4.126:151]
```
  [% IF eval.errormsg %]
```
[4.126]
[4.151]
```
  [% IF eval.evaluationerror.errormsg %]
```
Replacement in src/root/jobset-eval.tt at line 111 [17.4844]
B:BD[18.251] → [3.0:25]
```
  [% IF eval.errormsg %]
```
[18.251]
[3.25]
```
  [% IF eval.evaluationerror.errormsg %]
```

Replacement in src/root/jobset-eval.tt at line 113 [17.4844]

B:BD[3.69] → [3.69:252]

      <p>Errors occurred at [% INCLUDE renderDateTime timestamp=(eval.errortime || eval.timestamp) %].</p>
      <pre class="alert alert-error">[% HTML.escape(eval.errormsg) %]</pre>

[3.69]

[3.252]

      <p>Errors occurred at [% INCLUDE renderDateTime timestamp=(eval.evaluationerror.errortime || eval.timestamp) %].</p>
      <pre class="alert alert-error">[% HTML.escape(eval.evaluationerror.errormsg) %]</pre>

Replacement in src/root/jobset-eval.tt at line 175 [17.4844]
B:BD[19.4706] → [4.275:300]
```
  [% IF eval.errormsg %]
```
[19.4706]
[4.300]
```
  [% IF eval.evaluationerror.errormsg %]
```

Replacement in src/root/jobset-eval.tt at line 177 [17.4844]

B:BD[4.344] → [4.344:527]

      <p>Errors occurred at [% INCLUDE renderDateTime timestamp=(eval.errortime || eval.timestamp) %].</p>
      <pre class="alert alert-error">[% HTML.escape(eval.errormsg) %]</pre>

[4.344]

[4.527]

      <p>Errors occurred at [% INCLUDE renderDateTime timestamp=(eval.evaluationerror.errortime || eval.timestamp) %].</p>
      <pre class="alert alert-error">[% HTML.escape(eval.evaluationerror.errormsg) %]</pre>

Insertion in src/script/hydra-eval-jobset at line 695 [20.1319]

[2.612]


    my $evaluationErrorRecord = $db->resultset('EvaluationErrors')->create(
        { errormsg => $evaluationErrorMsg
        , errortime => $evaluationErrorTime
        }
    );

Insertion in src/script/hydra-eval-jobset at line 724 [20.1319]
[11.2572]
[14.6059]
Insertion in src/script/hydra-eval-jobset at line 727 [20.1319]
[14.6138]
[14.6138]
```
            , evaluationerror => $evaluationErrorRecord
```

Deletion in src/script/hydra-eval-jobset at line 730 [20.1319]

B:BD[21.853] → [2.614:708]

            , errormsg => $evaluationErrorMsg
            , errortime => $evaluationErrorTime

Insertion in src/sql/hydra.sql at line 440 [20.1]

[22.3210]

create table EvaluationErrors (
    id            serial primary key not null,
    errorMsg      text,    -- error output from the evaluator
    errorTime     integer  -- timestamp associated with errorMsg
);

Replacement in src/sql/hydra.sql at line 450 [20.1]

B:BD[23.37681] → [12.381:505]

    errorMsg      text, -- error output from the evaluator
    errorTime     integer, -- timestamp associated with errorMsg

[14.6713]

[12.505]

    evaluationerror_id integer,

Replacement in src/sql/hydra.sql at line 478 [20.1]

B:BD[14.7668] → [5.736:807]

    foreign key   (jobset_id) references Jobsets(id) on delete cascade

[14.7668]

[24.3974]

    foreign key   (jobset_id) references Jobsets(id) on delete cascade,
    foreign key   (evaluationerror_id) references EvaluationErrors(id) on delete set null

Insertion in src/sql/update-dbix.pl at line 24 [26.1755]
[25.10493]
[25.10493]
```
            "evaluationerrors" => "EvaluationErrors",
```

File addition: upgrade-73.sql (----------)

[9.3004]

create table EvaluationErrors (
    id            serial primary key not null,
    errorMsg      text,    -- error output from the evaluator
    errorTime     integer, -- timestamp associated with errorMsg
    jobsetEvalId  integer not null,
    FOREIGN KEY (jobsetEvalId)
        REFERENCES JobsetEvals(id)
        ON DELETE SET NULL
);
ALTER TABLE JobsetEvals
    ADD COLUMN evaluationerror_id integer NULL,
    ADD FOREIGN KEY (evaluationerror_id)
        REFERENCES EvaluationErrors(id)
        ON DELETE SET NULL;
INSERT INTO EvaluationErrors
    (errorMsg, errorTime, jobsetEvalId)
SELECT errorMsg, errorTime, id
FROM JobsetEvals
WHERE JobsetEvals.errorMsg != '' and JobsetEvals.errorMsg is not null;
UPDATE JobsetEvals
SET evaluationerror_id = EvaluationErrors.id
FROM EvaluationErrors
WHERE JobsetEvals.id = EvaluationErrors.jobsetEvalId
AND JobsetEvals.errorMsg != '' and JobsetEvals.errorMsg is not null;
ALTER TABLE JobsetEvals
    DROP COLUMN errorMsg,
    DROP COLUMN errorTime;
ALTER TABLE EvaluationErrors
    DROP COLUMN jobsetEvalId;

Move evaluation errors from evaluations to EvaluationErrors, a new table

Dependencies

In channels

Change contents

Replacement in src/lib/Hydra/Helper/Nix.pm at line 214 [8.436]

File addition: EvaluationErrors.pm (----------)

Deletion in src/lib/Hydra/Schema/JobsetEvals.pm at line 50 [11.65]

Replacement in src/lib/Hydra/Schema/JobsetEvals.pm at line 51 [11.65]

Insertion in src/lib/Hydra/Schema/JobsetEvals.pm at line 54 [11.65]

Replacement in src/lib/Hydra/Schema/JobsetEvals.pm at line 119 [11.65]

Insertion in src/lib/Hydra/Schema/JobsetEvals.pm at line 156 [11.65]

Insertion in src/lib/Hydra/Schema/JobsetEvals.pm at line 163 [11.65]

Replacement in src/lib/Hydra/Schema/JobsetEvals.pm at line 223 [11.65]

Replacement in src/root/common.tt at line 479 [16.2803]

Replacement in src/root/jobset-eval.tt at line 92 [17.4844]

Replacement in src/root/jobset-eval.tt at line 111 [17.4844]

Replacement in src/root/jobset-eval.tt at line 113 [17.4844]

Replacement in src/root/jobset-eval.tt at line 175 [17.4844]