|
2019-03-05
ยง
|
| 14:33 |
<jiji@cumin1001> |
conftool action : set/weight=5; selector: dc=codfw,service=citoid,cluster=scb,name=kubernetes.* |
[production] |
| 14:27 |
<hashar@deploy1001> |
scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="cawikibooks" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.JrfRQw0oDJ" --verbose' returned non-zero exit status 1 (duration: 00m 21s) |
[production] |
| 14:27 |
<hashar@deploy1001> |
Started scap: testwiki to php-1.33.0-wmf.20 and rebuild l10n cache # T206674 |
[production] |
| 14:25 |
<hashar@deploy1001> |
Pruned MediaWiki: 1.33.0-wmf.14 (duration: 09m 47s) |
[production] |
| 14:20 |
<otto@deploy1001> |
scap-helm eventgate-analytics finished |
[production] |
| 14:20 |
<otto@deploy1001> |
scap-helm eventgate-analytics cluster codfw completed |
[production] |
| 14:20 |
<otto@deploy1001> |
scap-helm eventgate-analytics cluster eqiad completed |
[production] |
| 14:20 |
<otto@deploy1001> |
scap-helm eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad,codfw] |
[production] |
| 14:17 |
<hashar@deploy1001> |
scap failed: LockFailedError Failed to acquire lock "/var/lock/scap.operations_mediawiki-config.lock"; owner is "hashar"; reason is "Pruned MediaWiki: 1.33.0-wmf.14" (duration: 00m 00s) |
[production] |
| 14:14 |
<hashar> |
Applied wmf/1.33.0-wmf.20 local patches # T206674 |
[production] |
| 14:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1064 T217591 (duration: 01m 50s) |
[production] |
| 13:31 |
<hashar> |
Cutting branch wmf/1.33.0-wmf.20 # T206674 |
[production] |
| 13:16 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1064 T217591 (duration: 00m 48s) |
[production] |
| 13:14 |
<ema> |
lvs500[12]: upgrade linux to 4.9.144-3.1, reboot for L1TF kernel/microcode updates T203011 |
[production] |
| 13:07 |
<zeljkof> |
EU SWAT finished |
[production] |
| 12:58 |
<zfilipin@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:487115|Set wgArticleCountMethod=any for zhwikiversity (T214946)]] (duration: 00m 49s) |
[production] |
| 12:45 |
<kartik@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Revert "Enable edittag for ExternalGuidance in CX and VE" (duration: 00m 48s) |
[production] |
| 12:24 |
<kartik@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Revert gerrit:493155 (duration: 00m 49s) |
[production] |
| 11:59 |
<_joe_> |
upgrading scap everywhere to 3.9.2-1, T217611 |
[production] |
| 11:52 |
<ema> |
lvs400[56]: upgrade linux to 4.9.144-3.1, reboot for L1TF kernel/microcode updates T203011 |
[production] |
| 11:45 |
<_joe_> |
installing new scap version in codfw |
[production] |
| 11:44 |
<oblivian@deploy1001> |
Synchronized README: Test deploy for new scap version (duration: 00m 48s) |
[production] |
| 11:43 |
<_joe_> |
installing new swat version on deployment servers, T217611 |
[production] |
| 11:22 |
<_joe_> |
uploading new scap packages , T217611 |
[production] |
| 10:58 |
<ema> |
lvs4007/lvs5003: upgrade linux to 4.9.144-3.1, reboot for L1TF kernel/microcode updates T203011 |
[production] |
| 10:57 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1103:3312 and db1103:3314 after mysql upgrade (duration: 00m 47s) |
[production] |
| 10:55 |
<gilles@deploy1001> |
Synchronized php-1.33.0-wmf.19/extensions/NavigationTiming/NavigationTiming.config.php: T187299 Fix wiki oversampling config validation (duration: 00m 48s) |
[production] |
| 10:37 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1103:3312 and db1103:3314 after mysql upgrade (duration: 00m 48s) |
[production] |
| 10:27 |
<jiji@cumin1001> |
conftool action : set/weight=4; selector: dc=eqiad,service=citoid,cluster=scb,name=kubernetes.* |
[production] |
| 10:24 |
<jijiki> |
Rump up citoid traffic from k8s to 25% - T213194 |
[production] |
| 10:18 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1103:3312 and db1103:3314 after mysql upgrade (duration: 00m 47s) |
[production] |
| 10:10 |
<gilles@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T187299 Oversample navtiming on ruwiki and eswiki (duration: 00m 47s) |
[production] |
| 10:07 |
<gilles@deploy1001> |
Synchronized php-1.33.0-wmf.19/extensions/NavigationTiming: T187299 Backport wiki oversampling config syntax change (duration: 00m 48s) |
[production] |
| 10:00 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1103:3312 and db1103:3314 after mysql upgrade (duration: 00m 50s) |
[production] |
| 09:56 |
<ema> |
lvs200[456]: upgrade linux to 4.9.144-3.1, reboot for L1TF kernel/microcode updates T203011 |
[production] |
| 09:31 |
<marostegui> |
Stop MySQL on db1103:3312 and db1103:3314 for MySQL upgrade |
[production] |
| 09:29 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1103:3312 and db1103:3314 for mysql upgrade (duration: 00m 47s) |
[production] |
| 09:26 |
<ema> |
lvs100[456]: reboot for L1TF kernel/microcode updates T203011 |
[production] |
| 09:23 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1081 (duration: 00m 47s) |
[production] |
| 09:16 |
<godog> |
kibana refresh field list |
[production] |
| 08:58 |
<mutante> |
restarting gerrit to pickup change 493963 - disable jgit gc |
[production] |
| 08:47 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1081 (duration: 00m 47s) |
[production] |
| 08:43 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1084 (duration: 00m 48s) |
[production] |
| 08:33 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1084 in API (duration: 00m 48s) |
[production] |
| 08:32 |
<marostegui> |
Optimize echo_event table on x1 codfw master (db2034) this will generate lag on x1 codfw - T217591 |
[production] |
| 08:24 |
<akosiaris> |
T213194 bump percentage of citoid requests reaching eqiad kubernetes cluster to 9% |
[production] |
| 08:23 |
<jiji@cumin1001> |
conftool action : set/pooled=yes; selector: dc=eqiad,service=citoid,cluster=scb,name=kubernetes100.* |
[production] |
| 08:14 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1084 (duration: 00m 49s) |
[production] |
| 07:47 |
<marostegui> |
Upgrade MySQL on db1084 |
[production] |
| 07:18 |
<marostegui> |
Stop MySQL on db1095 (backups host) to upgrade MySQL |
[production] |