551-600 of 10000 results (58ms)
2018-10-02 ยง
18:24 <jynus> restarting ferm on dbstore2002 T205257 [production]
18:07 <arlolra> Updated Parsoid to 65d6f82 (T163438, T205674, T205673) [production]
18:07 <ariel@deploy1001> Finished deploy [dumps/dumps@a9570fb]: fix incr dumps multiversion conf setting (duration: 00m 06s) [production]
18:07 <ariel@deploy1001> Started deploy [dumps/dumps@a9570fb]: fix incr dumps multiversion conf setting [production]
18:01 <arlolra@deploy1001> Finished deploy [parsoid/deploy@19053a3]: Updating Parsoid to 65d6f82 (duration: 10m 44s) [production]
17:51 <arlolra@deploy1001> Started deploy [parsoid/deploy@19053a3]: Updating Parsoid to 65d6f82 [production]
17:37 <XioNoX> update NAT for frbast2001 on pfw3-codfw - T204271 [production]
17:25 <XioNoX> update fw policies on pfw3-eqiad - T204271 [production]
17:22 <XioNoX> update fw policies on pfw3-codfw - T204271 [production]
17:22 <andrewbogott> upgraded wikitech-static to remotes/origin/REL1_31 [production]
17:18 <andrewbogott> upgrading debian packages and MediaWiki version on wikitech-static [production]
16:53 <jynus> setup test s3 replication channel on db1110 (filtered) [production]
16:49 <XioNoX> assign 10.195.0.129/29 to pfw3-codfw:reth0.2133 - T204271 [production]
16:38 <cmjohnson1> swapping failed disk db1067 T205780 [production]
16:04 <ppchelko@deploy1001> Finished deploy [cpjobqueue/deploy@093551f]: Increase cirrusSearchLinksUpdate concurrency (duration: 01m 06s) [production]
16:03 <ppchelko@deploy1001> Started deploy [cpjobqueue/deploy@093551f]: Increase cirrusSearchLinksUpdate concurrency [production]
15:50 <marxarelli> cutting 1.32.0-wmf.24 branch [production]
15:33 <gehel> cleanup old cronjob (cleanup GC logs) on all elasticsearch servers [production]
15:24 <akosiaris> upgrade mathoid chart version to 0.0.11 [production]
15:24 <akosiaris@deploy1001> scap-helm mathoid finished [production]
15:23 <akosiaris@deploy1001> scap-helm mathoid cluster codfw completed [production]
15:23 <akosiaris@deploy1001> scap-helm mathoid cluster eqiad completed [production]
15:23 <akosiaris@deploy1001> scap-helm mathoid upgrade production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] [production]
15:21 <akosiaris@deploy1001> scap-helm mathoid finished [production]
15:21 <akosiaris@deploy1001> scap-helm mathoid cluster codfw completed [production]
15:21 <akosiaris@deploy1001> scap-helm mathoid cluster eqiad completed [production]
15:21 <akosiaris@deploy1001> scap-helm mathoid upgrade -h [namespace: mathoid, clusters: eqiad,codfw] [production]
14:11 <banyek> powering off dbstore2002.codfw.wmnet for BBU change (T205257) [production]
13:46 <marostegui> Deploy schema change on s4 eqiad, this will generate lag on eqiad - T205913 [production]
13:06 <marostegui> Deploy schema change on s7 eqiad, this will generate lag on eqiad - T205913 [production]
12:47 <banyek> converting enwiki.content to TokuDB on host dbstrore1002 (T205544) [production]
12:47 <banyek> converting enwiki.contents to TokuDB on host dbstrore1002 (T205544) [production]
11:58 <banyek> converting wikidatawiki.slots to TokuDB on host dbstrore1002 (T205544) [production]
11:41 <arturo> downtime labstore1007 load check in icinga for 1d [production]
11:21 <zeljkof> EU SWAT finished [production]
11:19 <ladsgroup@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/FlaggedRevs/frontend/specialpages/reports/ProblemChanges_body.php: SWAT: [[gerrit:463917|Use proper index on change_tag table (T205904)]] (duration: 00m 57s) [production]
10:58 <mobrovac@deploy1001> Synchronized rpc/RunSingleJob.php: RunSingleJob: Delay job execution while in read-only mode - T204154 (duration: 00m 57s) [production]
10:34 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2092 (duration: 00m 56s) [production]
10:24 <marostegui> Deploy schema change on db2092 - T203709 [production]
10:24 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2092 (duration: 00m 56s) [production]
09:30 <marostegui> Deploy schema change on s2 eqiad master, lag will be generated T205913 [production]
08:43 <banyek> disabling puppet on es2001 and disabling backups too [production]
08:28 <marostegui> Deploy schema change on s6 eqiad master, lag will be generated T205913 [production]
08:16 <jynus> test recover some s3 wiki data onto db1110 (s5) [production]
08:04 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1110 (duration: 00m 56s) [production]
08:04 <marostegui> Deploy schema change on s5 eqiad master, lag will be generated T205913 [production]
08:01 <banyek> converting wikidatawiki.content to TokuDB on host dbstrore1002 (T205544) [production]
07:54 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2071 (duration: 00m 55s) [production]
07:50 <marostegui> Deploy schema change on db2071 T205913 [production]
07:50 <mholloway-shell@deploy1001> Finished deploy [tilerator/deploy@6c80537] (maps1004): Disable event logging requests and remove HTTP proxy (duration: 00m 17s) [production]