9251-9300 of 10000 results (82ms)
2019-03-05 ยง
14:34 <hashar@deploy1001> scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="cawikibooks" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.ngh6XIMz8y" --verbose' returned non-zero exit status 1 (duration: 00m 21s) [production]
14:33 <hashar@deploy1001> Started scap: testwiki to php-1.33.0-wmf.20 and rebuild l10n cache # T206674 [production]
14:33 <jiji@cumin1001> conftool action : set/weight=5; selector: dc=codfw,service=citoid,cluster=scb,name=kubernetes.* [production]
14:27 <hashar@deploy1001> scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="cawikibooks" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.JrfRQw0oDJ" --verbose' returned non-zero exit status 1 (duration: 00m 21s) [production]
14:27 <hashar@deploy1001> Started scap: testwiki to php-1.33.0-wmf.20 and rebuild l10n cache # T206674 [production]
14:25 <hashar@deploy1001> Pruned MediaWiki: 1.33.0-wmf.14 (duration: 09m 47s) [production]
14:20 <otto@deploy1001> scap-helm eventgate-analytics finished [production]
14:20 <otto@deploy1001> scap-helm eventgate-analytics cluster codfw completed [production]
14:20 <otto@deploy1001> scap-helm eventgate-analytics cluster eqiad completed [production]
14:20 <otto@deploy1001> scap-helm eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad,codfw] [production]
14:17 <hashar@deploy1001> scap failed: LockFailedError Failed to acquire lock "/var/lock/scap.operations_mediawiki-config.lock"; owner is "hashar"; reason is "Pruned MediaWiki: 1.33.0-wmf.14" (duration: 00m 00s) [production]
14:14 <hashar> Applied wmf/1.33.0-wmf.20 local patches # T206674 [production]
14:12 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1064 T217591 (duration: 01m 50s) [production]
13:31 <hashar> Cutting branch wmf/1.33.0-wmf.20 # T206674 [production]
13:16 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1064 T217591 (duration: 00m 48s) [production]
13:14 <ema> lvs500[12]: upgrade linux to 4.9.144-3.1, reboot for L1TF kernel/microcode updates T203011 [production]
13:07 <zeljkof> EU SWAT finished [production]
12:58 <zfilipin@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:487115|Set wgArticleCountMethod=any for zhwikiversity (T214946)]] (duration: 00m 49s) [production]
12:45 <kartik@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: Revert "Enable edittag for ExternalGuidance in CX and VE" (duration: 00m 48s) [production]
12:24 <kartik@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: Revert gerrit:493155 (duration: 00m 49s) [production]
11:59 <_joe_> upgrading scap everywhere to 3.9.2-1, T217611 [production]
11:52 <ema> lvs400[56]: upgrade linux to 4.9.144-3.1, reboot for L1TF kernel/microcode updates T203011 [production]
11:45 <_joe_> installing new scap version in codfw [production]
11:44 <oblivian@deploy1001> Synchronized README: Test deploy for new scap version (duration: 00m 48s) [production]
11:43 <_joe_> installing new swat version on deployment servers, T217611 [production]
11:22 <_joe_> uploading new scap packages , T217611 [production]
10:58 <ema> lvs4007/lvs5003: upgrade linux to 4.9.144-3.1, reboot for L1TF kernel/microcode updates T203011 [production]
10:57 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1103:3312 and db1103:3314 after mysql upgrade (duration: 00m 47s) [production]
10:55 <gilles@deploy1001> Synchronized php-1.33.0-wmf.19/extensions/NavigationTiming/NavigationTiming.config.php: T187299 Fix wiki oversampling config validation (duration: 00m 48s) [production]
10:37 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1103:3312 and db1103:3314 after mysql upgrade (duration: 00m 48s) [production]
10:27 <jiji@cumin1001> conftool action : set/weight=4; selector: dc=eqiad,service=citoid,cluster=scb,name=kubernetes.* [production]
10:24 <jijiki> Rump up citoid traffic from k8s to 25% - T213194 [production]
10:18 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1103:3312 and db1103:3314 after mysql upgrade (duration: 00m 47s) [production]
10:10 <gilles@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T187299 Oversample navtiming on ruwiki and eswiki (duration: 00m 47s) [production]
10:07 <gilles@deploy1001> Synchronized php-1.33.0-wmf.19/extensions/NavigationTiming: T187299 Backport wiki oversampling config syntax change (duration: 00m 48s) [production]
10:00 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1103:3312 and db1103:3314 after mysql upgrade (duration: 00m 50s) [production]
09:56 <ema> lvs200[456]: upgrade linux to 4.9.144-3.1, reboot for L1TF kernel/microcode updates T203011 [production]
09:31 <marostegui> Stop MySQL on db1103:3312 and db1103:3314 for MySQL upgrade [production]
09:29 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1103:3312 and db1103:3314 for mysql upgrade (duration: 00m 47s) [production]
09:26 <ema> lvs100[456]: reboot for L1TF kernel/microcode updates T203011 [production]
09:23 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1081 (duration: 00m 47s) [production]
09:16 <godog> kibana refresh field list [production]
08:58 <mutante> restarting gerrit to pickup change 493963 - disable jgit gc [production]
08:47 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1081 (duration: 00m 47s) [production]
08:43 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1084 (duration: 00m 48s) [production]
08:33 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1084 in API (duration: 00m 48s) [production]
08:32 <marostegui> Optimize echo_event table on x1 codfw master (db2034) this will generate lag on x1 codfw - T217591 [production]
08:24 <akosiaris> T213194 bump percentage of citoid requests reaching eqiad kubernetes cluster to 9% [production]
08:23 <jiji@cumin1001> conftool action : set/pooled=yes; selector: dc=eqiad,service=citoid,cluster=scb,name=kubernetes100.* [production]
08:14 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1084 (duration: 00m 49s) [production]