|
2019-06-04
ยง
|
| 12:57 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:22 |
<Urbanecm> |
ran mwscript deleteBatch.php --wiki=sawikisource -r '[[:phab:T214553|T214553]]: deleting useless red |
[production] |
| 12:13 |
<akosiaris> |
restart pybal on lvs2003, lvs1015 for sessionstore LVS configuration. T220401 |
[production] |
| 12:09 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2046 (duration: 00m 46s) |
[production] |
| 12:04 |
<akosiaris> |
restart pybal on lvs2006 for sessionstore LVS configuration. T220401 |
[production] |
| 11:40 |
<akosiaris> |
restart pybal on lvs1015 for sessionstore LVS configuration. T220401 |
[production] |
| 11:39 |
<krinkle@deploy1001> |
Synchronized php-1.34.0-wmf.7/includes/: T221577 / 1286d131c01886 (duration: 01m 04s) |
[production] |
| 11:39 |
<jijiki> |
enabling puppet on mc1* |
[production] |
| 11:38 |
<Urbanecm> |
run mwscript namespaceDupes.php --wiki=kuwiktionary --fix (T224327) |
[production] |
| 11:37 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:514239|Custom namespaces for ku.wiktionary]] (T224327) (duration: 00m 46s) |
[production] |
| 11:35 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:507931|Add localized project logo for sahwikiquote]] (2/2, T222065) (duration: 00m 47s) |
[production] |
| 11:33 |
<urbanecm@deploy1001> |
Synchronized static/images/project-logos/: [[:gerrit:507931|Add localized project logo for sahwikiquote]] (1/2, T222065) (duration: 00m 47s) |
[production] |
| 11:31 |
<jijiki> |
enabling puppet on mc2* |
[production] |
| 11:29 |
<Urbanecm> |
running mwscript namespaceDupes.php --wiki=sawikisource --add-prefix=T214553 --fix (T214553) |
[production] |
| 11:28 |
<Urbanecm> |
run mwscript namespaceDupes.php --wiki=thwiki --fix (T216322) |
[production] |
| 11:28 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:486221|Add Author namespace in Sanskrit Wikisource]] (T214553) (duration: 00m 46s) |
[production] |
| 11:24 |
<urbanecm@deploy1001> |
Synchronized wmf-config/CommonSettings.php: [[:gerrit:495918|Create new protection levels for dewiktionary]] (2/2, T216885) (duration: 00m 47s) |
[production] |
| 11:23 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:495918|Create new protection levels for dewiktionary]] (1/2, T216885) (duration: 00m 47s) |
[production] |
| 11:19 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:494016|Add editcontentmodel right to the templateeditor group on testwiki]] (T217499) (duration: 00m 47s) |
[production] |
| 11:15 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:491054|Add new namespaces for th.wiki]] (T216322) (duration: 00m 47s) |
[production] |
| 11:09 |
<krinkle@deploy1001> |
Synchronized php-1.34.0-wmf.6/includes/: T221577 / 1286d131c01886 (duration: 01m 07s) |
[production] |
| 11:02 |
<moritzm> |
draining ganeti1007 for eventual reboot to MDS-enabled Linux kernel |
[production] |
| 11:02 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 11:02 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 10:44 |
<jbond42> |
mw1* restarts will be delayed untill 11:15 |
[production] |
| 10:42 |
<jbond42> |
will start rolling reboots of mw1* servers 1t 10:50 |
[production] |
| 09:27 |
<moritzm> |
draining ganeti1006 for eventual reboot to MDS-enabled Linux kernel |
[production] |
| 09:25 |
<jijiki> |
disable puppet on mc* hosts to merge 511963 and 511973 |
[production] |
| 09:01 |
<moritzm> |
draining ganeti1005 for eventual reboot to MDS-enabled Linux kernel |
[production] |
| 08:57 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 08:57 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 08:32 |
<elukey> |
remove memcached nutcracker config from mw1* hosts (not used). Changes will be picked up when nutcracker will be restarted (after reboots, etc..) - T214275 |
[production] |
| 08:23 |
<moritzm> |
draining ganeti1004 for eventual reboot to MDS-enabled Linux kernel |
[production] |
| 08:04 |
<marostegui> |
Stop MySQL on db2046 to clone db2058 - T221533 |
[production] |
| 08:04 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Depool db2046 (duration: 00m 47s) |
[production] |
| 08:03 |
<elukey> |
restart hive-server2 on an-coord1001 to pick up new GC/Heap settings |
[production] |
| 07:35 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@abcb534]: Use only Proton for PDF rendering - T210651 (duration: 19m 16s) |
[production] |
| 07:22 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 07:21 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 07:21 |
<moritzm> |
draining ganeti1002 for eventual reboot to MDS-enabled Linux kernel |
[production] |
| 07:16 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Move db2058 from s4 to s6 (duration: 00m 47s) |
[production] |
| 07:16 |
<mobrovac@deploy1001> |
Started deploy [restbase/deploy@abcb534]: Use only Proton for PDF rendering - T210651 |
[production] |
| 06:57 |
<elukey> |
restart hive metastore on an-coord1001 to apply new GC/heap settings |
[production] |
| 06:42 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1091 after upgrade (duration: 00m 48s) |
[production] |
| 06:21 |
<elukey> |
restart pdfrender on scb1002 (flapping) |
[production] |
| 06:04 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1091 after upgrade (duration: 00m 47s) |
[production] |
| 05:54 |
<marostegui> |
Stop MySQL on db2078:m3 - T221533 |
[production] |
| 05:51 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1091 after upgrade (duration: 00m 47s) |
[production] |
| 05:40 |
<marostegui> |
Stop MySQL on db1091 for MySQL upgrade T224852 |
[production] |
| 05:40 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1091 for upgrade (duration: 00m 48s) |
[production] |