|
2019-06-05
ยง
|
| 18:20 |
<mforns> |
finished deployment of analytics/refinery/source v0.0.91 |
[analytics] |
| 18:16 |
<andrewbogott> |
depooling and moving tools-sgeexec-0921 and tools-sgeexec-0929 |
[tools] |
| 18:00 |
<mforns> |
starting deployment of analytics/refinery/source v0.0.91 |
[analytics] |
| 17:36 |
<marostegui> |
Start replication db1091 - T225060 |
[production] |
| 17:35 |
<James_F> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/514547 |
[releng] |
| 17:32 |
<marostegui> |
Start MySQL with replication stopped on db1091 - T225060 |
[production] |
| 16:29 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Revert user-blocks-change to use eventbus and old schema - T211248 (duration: 00m 54s) |
[production] |
| 16:22 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: use eventgate-main for 2 events on all wikis - T211248 (duration: 00m 55s) |
[production] |
| 16:11 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Add wgEventServiceStreamConfig and switch 2 topics in group0 T222822 (duration: 00m 56s) |
[production] |
| 16:11 |
<XioNoX> |
remove BGP to AS38082 on cr4-ulsfo (left the IXP) |
[production] |
| 16:01 |
<James_F> |
Reloading Zuul to deploy 172a4e7886b285adb48a85dce7287dce5c3778e2 |
[releng] |
| 15:46 |
<reedy@deploy1001> |
Scap failed!: Call to mwscript eval.php returned: None |
[production] |
| 15:44 |
<reedy@deploy1001> |
Finished scap: Rebuild .8 i18n for FlaggedRevs (duration: 41m 14s) |
[production] |
| 15:36 |
<moritzm> |
installing exim4 security updates |
[production] |
| 15:03 |
<wm-bot> |
<jeanfred> Deploy latest from Git master: 5317bb7, aabd6ec (T224862) |
[tools.wikiloves] |
| 15:03 |
<reedy@deploy1001> |
Started scap: Rebuild .8 i18n for FlaggedRevs |
[production] |
| 14:24 |
<marostegui> |
Poweroff db1091 for BBU replacement - T225060 |
[production] |
| 13:57 |
<elukey> |
restart mcrouter on MediaWiki app/api canaries to pick up new config change (timeouts before marking a memcached shard as TKO from 3 to 10) - T203786 |
[production] |
| 13:56 |
<jijiki> |
enabling puppet and pooling on mw* canaries |
[production] |
| 13:17 |
<jynus> |
start es2,es3 backup on codfw |
[production] |
| 13:17 |
<zfilipin@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.34.0-wmf.8 |
[production] |
| 13:03 |
<hashar> |
restarting Jenkins |
[production] |
| 12:53 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1135 (duration: 00m 54s) |
[production] |
| 12:46 |
<Lucas_WMDE> |
EU SWAT finished |
[production] |
| 12:40 |
<arturo> |
rebase git repos in toolsbeta-puppetmaster-02. There was some rebase problems in labs/private that required me re-creating by hand one of the [local] patches (puppetdb secrets) |
[toolsbeta] |
| 12:33 |
<arturo> |
drop VM instances toolsbeta-k8s-master-arturo-[1-3] and create toolsbeta-arturo-k8s-master-[1-3] T215531 |
[toolsbeta] |
| 12:32 |
<ladsgroup@deploy1001> |
Synchronized php-1.34.0-wmf.8/extensions/WikimediaMessages/: SWAT: [[gerrit:514460|Fix wikidata copyright message (T224536)]] (duration: 00m 56s) |
[production] |
| 12:32 |
<arturo> |
drop puppet prefix `toolsbeta-k8s-master-arturo` and create `toolsbeta-arturo-k8s-master` since there is also `toolsbeta-k8s-master` which get applied to my VMs T215531 |
[toolsbeta] |
| 11:43 |
<pmiazga@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:514449|Enable the new history page in the advanced mobile contributions mode (T219895)]] (duration: 00m 56s) |
[production] |
| 11:42 |
<arturo> |
create VM `toolsbeta-k8s-master-arturo-3` for T215531 (so I have 3 master nodes in this k8s deployment) |
[toolsbeta] |
| 11:38 |
<arturo> |
delete instances arturo-sgeexec-sssd-test-2, arturo-sgeexec-sssd-test-1, arturo-bastion-sssd-test, unused |
[toolsbeta] |
| 11:27 |
<urbanecm@deploy1001> |
Synchronized wmf-config/flaggedrevs.php: [[:gerrit:514413|Remove project namespace from flaggedrevs on ruwikisource]] (T225037) (duration: 00m 54s) |
[production] |
| 10:57 |
<ladsgroup@deploy1001> |
Synchronized php-1.34.0-wmf.8/extensions/FlaggedRevs: [[gerrit:514456|Add ext.flaggedRevs.icons to modules registeration]] (duration: 00m 57s) |
[production] |
| 10:14 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1135 (duration: 00m 55s) |
[production] |
| 10:09 |
<godog> |
mount sdb3 on ms-be1022 - T225079 |
[production] |
| 10:07 |
<elukey> |
attempt to re-run webrequest-load-wf-text-2019-6-4-20 via Hue (temporary errors in the logs) |
[analytics] |
| 09:50 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Pool db1135 with very low weight on s4 (duration: 00m 55s) |
[production] |
| 09:26 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Pool without traffic db1135 into s4 T225060 (duration: 00m 55s) |
[production] |
| 09:25 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Pool without traffic db1135 into s4 T225060 (duration: 00m 56s) |
[production] |
| 09:07 |
<hashar> |
Pooled in integration-slave-docker-1058 and integration-slave-docker-1059 |
[releng] |
| 08:56 |
<arturo> |
move integration-slave-docker-1059 and integration-slave-docker-1058 to cloudvirt1028 (T223971) |
[integration] |
| 08:42 |
<onimisionipe> |
removing maps2001 from cassandra cluster. It is going to be reimaged - T224395 |
[production] |
| 08:40 |
<_joe_> |
rolling restart of php7 on the api servers, to test a different strategy of restarting compared to the appservers. |
[production] |
| 08:21 |
<_joe_> |
performing a rolling restart of the php appservers via cumin to test speed and safety of the operations proposed in T224857 |
[production] |
| 08:13 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 08:12 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 08:12 |
<moritzm> |
rebooting pybal-test2001 for tests with new qemu |
[production] |
| 08:12 |
<ema> |
pool cp3035 w/ ATS backend T222937 |
[production] |
| 08:12 |
<marostegui> |
Reboot db1091 T225060 |
[production] |
| 08:05 |
<moritzm> |
installing qemu security updates on Ganeti hosts |
[production] |