|
2018-03-08
§
|
| 13:40 |
<elukey> |
eventlogging analytics migrated from eventlog1001 to eventlog1002 |
[production] |
| 13:35 |
<ariel@tin> |
Finished deploy [dumps/dumps@f26c114]: fix prefetch stubs; retrieval globals more robustly (duration: 00m 03s) |
[production] |
| 13:35 |
<ariel@tin> |
Started deploy [dumps/dumps@f26c114]: fix prefetch stubs; retrieval globals more robustly |
[production] |
| 13:29 |
<ema> |
cp-ulsfo: reboot for retpoline kernel updates T188092 |
[production] |
| 12:50 |
<oblivian@puppetmaster1001> |
conftool action : edit; selector: scope=codfw,name=ReadOnly |
[production] |
| 12:47 |
<oblivian@puppetmaster1001> |
conftool action : edit; selector: scope=codfw,name=ReadOnly |
[production] |
| 12:06 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1114 fully (duration: 01m 16s) |
[production] |
| 11:32 |
<moritzm> |
installing isc-dhcp security updates |
[production] |
| 10:43 |
<moritzm> |
installing libvpx security updates |
[production] |
| 10:22 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Change db1114 load (duration: 01m 16s) |
[production] |
| 10:14 |
<akosiaris> |
conduct IO stresstests on ganeti1005 (sca1004 VM) with cache=none KVM flag on T181121 |
[production] |
| 10:13 |
<akosiaris> |
conduct IO stresstests on ganeti1005 (sca1004 VM) with cache=none KVM flag on |
[production] |
| 09:57 |
<dcausse> |
restaring mjolnir-kafka-daemon.service on relforge1002 to switch to kafka jumbo |
[production] |
| 09:56 |
<dcausse> |
restaring mjolnir-kafka-daemon.service on relforge1001 to switch to kafka jumbo |
[production] |
| 09:56 |
<_joe_> |
decommissioning mw2017-2099 T187467 |
[production] |
| 09:46 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1114 partially (duration: 01m 16s) |
[production] |
| 09:44 |
<moritzm> |
rearming keyholder on neodymium after reboot |
[production] |
| 09:40 |
<moritzm> |
rebooting neodymium for kernel security update |
[production] |
| 09:22 |
<ema> |
cp-eqsin: reboot for retpoline kernel updates T188092 |
[production] |
| 09:12 |
<ema> |
cp3043: varnish-be-restart T189085 |
[production] |
| 09:08 |
<moritzm> |
rebooting bast1001 for kernel security update |
[production] |
| 08:58 |
<elukey> |
restart varnish backend on cp3041 (failed fetches) |
[production] |
| 08:58 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2046, db2053 and db2060 after kernel upgrade (duration: 01m 15s) |
[production] |
| 08:58 |
<moritzm> |
reset RAC on bast1001, serial console was stuck |
[production] |
| 08:50 |
<elukey> |
rebooting analytics1003 (Hadoop Hive, Oozie, etc..) for kernel updates |
[production] |
| 08:32 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Depool db2046, db2053 and db2060 for kernel upgrade (duration: 01m 17s) |
[production] |
| 08:31 |
<elukey> |
reboot analytics1002 (Hadoop master standby) for kernel upgrades |
[production] |
| 08:28 |
<marostegui> |
Stop MySQL on db2046, db2053 and db2060 for kernel upgrade |
[production] |
| 08:19 |
<elukey> |
reboot analytics1001 (Hadoop master) for kernel upgrade (temp failover to analytics1002) |
[production] |
| 08:09 |
<ema> |
cp3040: varnish-be-restart T189085 |
[production] |
| 08:00 |
<ema> |
cp3032: varnish-be-restart T189085 |
[production] |
| 07:44 |
<elukey> |
reboot kafka2003 (eventbus codfw) for kernel updates |
[production] |
| 07:24 |
<elukey> |
reboot kafka2002 (eventbus codfw) for kernel updates |
[production] |
| 07:22 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool es1019 for maintenance - T187530 (duration: 01m 16s) |
[production] |
| 07:15 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Revert: Depool db1064, it is not performing well with 2 failed disks - T188685 (duration: 01m 31s) |
[production] |
| 04:27 |
<Krinkle> |
Running whisper-mass-resize for ResourceLoader.* metrics on graphite1001 and graphite2001 (T179622) |
[production] |
| 02:30 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.23) (duration: 07m 37s) |
[production] |
| 02:15 |
<tgr@tin> |
Synchronized wmf-config/throttle.php: T189161 Temporarely remove account creation limit for event on Portuguese Wikipedia on March 08, 2018 (duration: 01m 10s) |
[production] |
| 01:17 |
<twentyafterfour> |
phabricator update completed |
[production] |
| 01:13 |
<twentyafterfour> |
preparing for phabricator update 2018-03-07/1 |
[production] |
| 00:37 |
<thcipriani@tin> |
Synchronized wmf-config/db-eqiad.php: SWAT: [[gerrit:417173|wikitech: use FQDNs for m5 cluster members]] (duration: 01m 16s) |
[production] |
| 00:28 |
<thcipriani@tin> |
Synchronized wmf-config/Wikibase-production.php: SWAT: [[gerrit:413899|Add configuration for CirrusSearch to instantly index new Wikidata items]] T183053 (duration: 01m 15s) |
[production] |
| 00:16 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:416631|Enable loginOnly mode for local auth provider on group 2]] T57420 (duration: 01m 16s) |
[production] |