2351-2400 of 10000 results (31ms)
2020-05-22 §
07:48 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1149 and db1081', diff saved to https://phabricator.wikimedia.org/P11276 and previous config saved to /var/cache/conftool/dbconfig/20200522-074853-marostegui.json [production]
07:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1149 and db1081', diff saved to https://phabricator.wikimedia.org/P11275 and previous config saved to /var/cache/conftool/dbconfig/20200522-072000-marostegui.json [production]
07:07 <elukey@puppetmaster1001> conftool action : set/pooled=yes:weight=10; selector: name=druid1008.eqiad.wmnet [production]
07:04 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=druid1007.eqiad.wmnet [production]
07:04 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=druid1007.eqiad.wmnet [production]
04:34 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1081 - T252512', diff saved to https://phabricator.wikimedia.org/P11272 and previous config saved to /var/cache/conftool/dbconfig/20200522-043418-marostegui.json [production]
2020-05-21 §
23:58 <ejegg> updated civicrm from b658fd8233 to 6b1d5902dd [production]
23:54 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.32/includes/content/ContentHandlerFactory.php: If578893f5689 (duration: 01m 06s) [production]
23:47 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.32/extensions/LiquidThreads/classes/Thread.php: If3418cba06e (duration: 01m 07s) [production]
23:41 <krinkle@deploy1001> Synchronized wmf-config/mc.php: I222457729a5b (duration: 01m 08s) [production]
21:46 <eileen> civicrm revision changed from ed4c9522ac to b658fd8233, config revision is 9babae3954 [production]
21:10 <foks> removing two files for legal compliance [production]
20:44 <bstorm_> labstore1005 is now running stretch and drbd devices are resyncing after several reboots and some significant effort T224582 [production]
18:24 <twentyafterfour> restarting phabricator on phab1001 to deploy https://phabricator.wikimedia.org/rPHEX2687d08786a9dadcbaa96709de991f471f239830 [production]
17:24 <bblack> anycast experiment done, all back to normal [production]
17:20 <bblack> anycast experimentation commencing in ulsfo (test route withdrawal)... [production]
17:04 <bstorm_> starting labstore1005 upgrades T224582 [production]
16:14 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:12 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:04 <sbassett@deploy1001> Synchronized private/PrivateSettings.php: Update mitigations for T250887 (duration: 01m 08s) [production]
15:48 <andrewbogott> rebuilding cloudnet1003.eqiad.wmnet with Debian Buster for T253124 [production]
15:22 <XioNoX> Add BGP between cr1/2-eqiad and authdns1001 - T253196 [production]
15:09 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:09 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:08 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:08 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:07 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:07 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:59 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw217[0-2].codfw.wmnet [production]
14:59 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw216[0-9].codfw.wmnet [production]
14:58 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw215[8-9].codfw.wmnet [production]
14:50 <bblack@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:47 <bblack@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:44 <akosiaris@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'mathoid' for release 'canary' . [production]
14:33 <akosiaris> upload helmfile 0.109.0 to apt.wikimedia.org/buster-wikimedia and stretch-wikimedia, component main [production]
13:51 <vgutierrez> depool cp4032 for some ats tests [production]
13:22 <mutante> cloudnet1004 - reboot to test PXE boot [production]
12:44 <andrewbogott> reimaging cloudnet1004.eqiad.wmnet for T253124 [production]
12:29 <elukey> roll restart druid-public cluster (druid100[4-6], backend for the AQS API) to apply new settings + openjdk upgrade - T252771 [production]
12:13 <mutante> depooled mw2158 through mw2172 to make room again in C3 as planned (T247018) [production]
12:12 <marostegui> Repool labsdb1011 into the analytics role 🤞- T249188 [production]
12:12 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw217[0-2].codfw.wmnet [production]
12:10 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw216[0-9].codfw.wmnet [production]
12:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1143 and db1091', diff saved to https://phabricator.wikimedia.org/P11270 and previous config saved to /var/cache/conftool/dbconfig/20200521-120555-marostegui.json [production]
12:05 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw215[8-9].codfw.wmnet [production]
11:18 <hnowlan> Removed changeprop from scb hosts [production]
11:04 <vgutierrez> rolling restart of ncredir servers for kernel update [production]
10:17 <vgutierrez> restart of acme-chief servers for kernel update [production]
10:13 <jbond42> deploy CI for pupet privcate repo [production]
10:11 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1143 and db1091', diff saved to https://phabricator.wikimedia.org/P11268 and previous config saved to /var/cache/conftool/dbconfig/20200521-101100-marostegui.json [production]