401-450 of 10000 results (44ms)
2018-10-02 §
15:24 <akosiaris@deploy1001> scap-helm mathoid finished [production]
15:23 <akosiaris@deploy1001> scap-helm mathoid cluster codfw completed [production]
15:23 <akosiaris@deploy1001> scap-helm mathoid cluster eqiad completed [production]
15:23 <akosiaris@deploy1001> scap-helm mathoid upgrade production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] [production]
15:21 <akosiaris@deploy1001> scap-helm mathoid finished [production]
15:21 <akosiaris@deploy1001> scap-helm mathoid cluster codfw completed [production]
15:21 <akosiaris@deploy1001> scap-helm mathoid cluster eqiad completed [production]
15:21 <akosiaris@deploy1001> scap-helm mathoid upgrade -h [namespace: mathoid, clusters: eqiad,codfw] [production]
14:11 <banyek> powering off dbstore2002.codfw.wmnet for BBU change (T205257) [production]
13:46 <marostegui> Deploy schema change on s4 eqiad, this will generate lag on eqiad - T205913 [production]
13:06 <marostegui> Deploy schema change on s7 eqiad, this will generate lag on eqiad - T205913 [production]
12:47 <banyek> converting enwiki.content to TokuDB on host dbstrore1002 (T205544) [production]
12:47 <banyek> converting enwiki.contents to TokuDB on host dbstrore1002 (T205544) [production]
11:58 <banyek> converting wikidatawiki.slots to TokuDB on host dbstrore1002 (T205544) [production]
11:41 <arturo> downtime labstore1007 load check in icinga for 1d [production]
11:21 <zeljkof> EU SWAT finished [production]
11:19 <ladsgroup@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/FlaggedRevs/frontend/specialpages/reports/ProblemChanges_body.php: SWAT: [[gerrit:463917|Use proper index on change_tag table (T205904)]] (duration: 00m 57s) [production]
10:58 <mobrovac@deploy1001> Synchronized rpc/RunSingleJob.php: RunSingleJob: Delay job execution while in read-only mode - T204154 (duration: 00m 57s) [production]
10:34 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2092 (duration: 00m 56s) [production]
10:24 <marostegui> Deploy schema change on db2092 - T203709 [production]
10:24 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2092 (duration: 00m 56s) [production]
09:30 <marostegui> Deploy schema change on s2 eqiad master, lag will be generated T205913 [production]
08:43 <banyek> disabling puppet on es2001 and disabling backups too [production]
08:28 <marostegui> Deploy schema change on s6 eqiad master, lag will be generated T205913 [production]
08:16 <jynus> test recover some s3 wiki data onto db1110 (s5) [production]
08:04 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1110 (duration: 00m 56s) [production]
08:04 <marostegui> Deploy schema change on s5 eqiad master, lag will be generated T205913 [production]
08:01 <banyek> converting wikidatawiki.content to TokuDB on host dbstrore1002 (T205544) [production]
07:54 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2071 (duration: 00m 55s) [production]
07:50 <marostegui> Deploy schema change on db2071 T205913 [production]
07:50 <mholloway-shell@deploy1001> Finished deploy [tilerator/deploy@6c80537] (maps1004): Disable event logging requests and remove HTTP proxy (duration: 00m 17s) [production]
07:49 <mholloway-shell@deploy1001> Started deploy [tilerator/deploy@6c80537] (maps1004): Disable event logging requests and remove HTTP proxy [production]
07:49 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2071 (duration: 00m 56s) [production]
07:48 <mholloway-shell@deploy1001> Finished deploy [kartotherian/deploy@0bf513a] (maps1004): Remove HTTP proxy (duration: 00m 16s) [production]
07:48 <mholloway-shell@deploy1001> Started deploy [kartotherian/deploy@0bf513a] (maps1004): Remove HTTP proxy [production]
07:42 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2088:3311 (duration: 00m 56s) [production]
07:36 <marostegui> Deploy schema change on db2088:3311 T205913 [production]
07:36 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2088:3311 (duration: 00m 55s) [production]
07:32 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2072 (duration: 00m 55s) [production]
07:18 <marostegui> Deploy schema change on db2072 T205913 [production]
07:17 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2072 (duration: 01m 02s) [production]
05:22 <_joe_> stopped tilerator on maps1004, was spamming like crazy [production]
01:18 <ejegg> updated CiviCRM from e7a620a00c to c353eba283 [production]
2018-10-01 §
23:44 <eileen> update process control revision is b9c7ab286e - define but not enable Redis [production]
23:43 <foks> disabling 2FA for two users [production]
23:31 <twentyafterfour> finished creating database tables [production]
23:18 <twentyafterfour> creating ipblocks_restrictions table (command run on mwmaint2001: foreachwiki sql.php maintenance/archives/patch-ipblocks_restrictions-table.sql) [production]
22:52 <ppchelko@deploy1001> Finished deploy [restbase/deploy@babfe80]: Don't log the request for transform failures, take 3, feeds check timeouts (duration: 06m 22s) [production]
22:46 <ppchelko@deploy1001> Started deploy [restbase/deploy@babfe80]: Don't log the request for transform failures, take 3, feeds check timeouts [production]
22:45 <ppchelko@deploy1001> Finished deploy [restbase/deploy@babfe80]: Don't log the request for transform failures, take 2, feeds check timeouts (duration: 03m 57s) [production]