2651-2700 of 10000 results (28ms)
2020-06-03 §
09:08 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:05 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:04 <marostegui> Reimage db1080 [production]
09:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1080 for reimage', diff saved to https://phabricator.wikimedia.org/P11376 and previous config saved to /var/cache/conftool/dbconfig/20200603-090143-marostegui.json [production]
08:42 <kormat@deploy1001> Synchronized wmf-config/db-codfw.php: Replace pc2009 with pc2010 while reimaging (duration: 01m 16s) [production]
08:19 <moritzm> upgrading mw1261 to PHP 7.2.31 [production]
08:16 <XioNoX> re-add ae2 physical interfaces to external group - T253970 [production]
08:09 <moritzm> upgrading remaining mwdebug* servers to PHP 7.2.31 [production]
08:08 <kormat> reimaging pc2009 to buster T252182 [production]
08:08 <XioNoX> remove ae2 physical interfaces from external group - T253970 [production]
07:52 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
07:51 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
07:48 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
07:45 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
07:44 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw218[0-6].codfw.wmnet [production]
07:36 <mutante> depooling mw2180 - mw2186 [production]
07:36 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw218[0-6].codfw.wmnet [production]
07:35 <moritzm> imported PHP 7.2.31 to apt.wikimedia.org/component/php72 [production]
07:33 <ema> cp: upgrade purged to 0.15 [production]
07:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2071 after cloning it from db2130 to restore all the schema changes applied', diff saved to https://phabricator.wikimedia.org/P11375 and previous config saved to /var/cache/conftool/dbconfig/20200603-072841-marostegui.json [production]
07:15 <XioNoX> repool esams - T254021 [production]
07:09 <XioNoX> re-activate peering/transit BGP on cr2-esams - T254021 [production]
07:00 <XioNoX> re0.cr2-esams> request system reboot both-routing-engines - T254021 [production]
06:56 <XioNoX> deactivate peering/transit BGP cr2-esams - T244497 [production]
06:54 <XioNoX> failover vrrp to cr3-esams - T244497 [production]
06:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1138 T253808', diff saved to https://phabricator.wikimedia.org/P11374 and previous config saved to /var/cache/conftool/dbconfig/20200603-063752-marostegui.json [production]
06:18 <XioNoX> cr3-esams> request chassis routing-engine master switch - T244497 [production]
06:11 <XioNoX> cr3-esams> request vmhost reboot re1 (backup re) - T244497 [production]
06:08 <XioNoX> re-activate transit BGP to cr3-knams - T254021 [production]
06:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1138 T253808', diff saved to https://phabricator.wikimedia.org/P11373 and previous config saved to /var/cache/conftool/dbconfig/20200603-060124-marostegui.json [production]
05:58 <XioNoX> reboot cr3-knams - T254021 [production]
05:51 <XioNoX> deactivate transit BGP ton cr3-knams - T254021 [production]
05:48 <XioNoX> depool esams - T254021 [production]
05:41 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2130 to clone db2071', diff saved to https://phabricator.wikimedia.org/P11371 and previous config saved to /var/cache/conftool/dbconfig/20200603-054117-marostegui.json [production]
05:40 <marostegui> Stop MySQL on db2130 to clone db2071 [production]
05:38 <XioNoX> deactivate graceful-switchover on cr3-esams - T254021 [production]
05:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1138 T253808', diff saved to https://phabricator.wikimedia.org/P11370 and previous config saved to /var/cache/conftool/dbconfig/20200603-053748-marostegui.json [production]
05:35 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
05:32 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
05:14 <XioNoX> turn cr1-codfw:fpc0 online - T254110 [production]
05:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1138 T253808', diff saved to https://phabricator.wikimedia.org/P11369 and previous config saved to /var/cache/conftool/dbconfig/20200603-050911-marostegui.json [production]
01:00 <reedy@deploy1001> Synchronized wmf-config/CommonSettings-labs.php: wgLocalVirtualHosts (duration: 01m 06s) [production]
00:59 <krinkle@deploy1001> Synchronized wmf-config/mc.php: Ic27b605aec0118c5 (duration: 01m 11s) [production]
2020-06-02 §
23:58 <ejegg> updated fundraising CiviCRM from 657c4b9455 to eb156dffa4 [production]
23:55 <ejegg> updated payments-wiki from 1942a537ef to e46114d8b1 [production]
22:48 <cstone> civicrm revision changed from d1cd99166f to 657c4b9455 [production]
21:48 <reedy@deploy1001> Synchronized wmf-config/interwiki-labs.php: laaaaabs (duration: 01m 05s) [production]
21:23 <reedy@deploy1001> Synchronized multiversion/MWMultiVersion.php: beta apiportalwiki T254185 (duration: 01m 06s) [production]
21:21 <reedy@deploy1001> Synchronized wmf-config/config/apiportalwiki.yaml: beta apiportalwiki T254185 (duration: 01m 05s) [production]
21:20 <reedy@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: beta apiportalwiki T254185 (duration: 01m 05s) [production]