1651-1700 of 10000 results (30ms)
2020-11-03 §
09:49 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:49 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:18 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:18 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:12 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:12 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:06 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:06 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:05 <kormat@cumin1001> dbctl commit (dc=all): 'Repooling after reboot. T261389', diff saved to https://phabricator.wikimedia.org/P13139 and previous config saved to /var/cache/conftool/dbconfig/20201103-090523-kormat.json [production]
09:00 <kormat@cumin1001> dbctl commit (dc=all): 'Rebooting for T261389', diff saved to https://phabricator.wikimedia.org/P13138 and previous config saved to /var/cache/conftool/dbconfig/20201103-090013-kormat.json [production]
09:00 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:00 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:32 <godog> Prometheus re-enable compactions - T261281 [production]
06:59 <marostegui> Remove db1091 from tendril and zarcillo T267088 [production]
06:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove db1091 from dbctl T267088', diff saved to https://phabricator.wikimedia.org/P13137 and previous config saved to /var/cache/conftool/dbconfig/20201103-065756-marostegui.json [production]
06:46 <marostegui> Deploy schema change on s1 codfw master: T265349 [production]
06:16 <marostegui> Stop MySQL on es1014 to clone es1028 T261717 [production]
06:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es1014 to reclone es1028 T261717', diff saved to https://phabricator.wikimedia.org/P13136 and previous config saved to /var/cache/conftool/dbconfig/20201103-061423-marostegui.json [production]
06:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Promote es1019 to es3 master (this is a noop) T261717', diff saved to https://phabricator.wikimedia.org/P13135 and previous config saved to /var/cache/conftool/dbconfig/20201103-061403-marostegui.json [production]
06:11 <marostegui> Stop MySQL on es1012 to clone es1027 T261717 [production]
06:07 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es1012 to reclone es1027 T261717', diff saved to https://phabricator.wikimedia.org/P13134 and previous config saved to /var/cache/conftool/dbconfig/20201103-060727-marostegui.json [production]
06:07 <marostegui@cumin1001> dbctl commit (dc=all): 'Promote es1018 to es1 master (this is a noop) T261717', diff saved to https://phabricator.wikimedia.org/P13133 and previous config saved to /var/cache/conftool/dbconfig/20201103-060705-marostegui.json [production]
06:04 <marostegui> Stop MySQL on es1011 to clone es1026 T261717 [production]
06:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es1011 to reclone es1026 T261717', diff saved to https://phabricator.wikimedia.org/P13132 and previous config saved to /var/cache/conftool/dbconfig/20201103-060054-marostegui.json [production]
06:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Promote es1015 to es2 master (this is a noop) T261717', diff saved to https://phabricator.wikimedia.org/P13131 and previous config saved to /var/cache/conftool/dbconfig/20201103-060038-marostegui.json [production]
04:39 <cstone> civicrm revision changed from cd13d9e30f to b1342c4129 [production]
02:13 <shdubsh> restart ES on logstash1009 - oom killed [production]
01:01 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
00:59 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
00:42 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
00:40 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
2020-11-02 §
22:19 <twentyafterfour> restart php7.3-fpm on phab1001 [production]
22:03 <twentyafterfour> applied 113a244a66 on phab1001 to hotfix T240862 [production]
20:22 <eileen> process-control config revision is 313a36312f re-enable thank you [production]
19:56 <dzahn@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:48 <dzahn@cumin1001> START - Cookbook sre.dns.netbox [production]
19:47 <eileen> civicrm revision changed from 3317d30356 to cd13d9e30f, config revision is db912e3bba [production]
19:45 <eileen> process-control config revision is db912e3bba - thankyou job off for testing [production]
19:07 <Urbanecm> Deployed security fix for T205908 [production]
19:04 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
18:59 <andrewbogott> added dcaro to ops and wmf ldap groups [production]
18:59 <mutante> decom'ing testvm1001 [production]
18:58 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
18:49 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
18:49 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
18:49 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
18:49 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
18:17 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
18:17 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
18:14 <XioNoX> push new pfw policies - T267051 [production]