2401-2450 of 10000 results (89ms)
2020-03-27 §
07:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1130 for schema change', diff saved to https://phabricator.wikimedia.org/P10795 and previous config saved to /var/cache/conftool/dbconfig/20200327-070224-marostegui.json [production]
07:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1082 after schema change', diff saved to https://phabricator.wikimedia.org/P10794 and previous config saved to /var/cache/conftool/dbconfig/20200327-070014-marostegui.json [production]
06:31 <marostegui> Deploy schema change on db1082, this will generate lag on s5 labs [production]
06:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1082 for schema change', diff saved to https://phabricator.wikimedia.org/P10793 and previous config saved to /var/cache/conftool/dbconfig/20200327-063042-marostegui.json [production]
2020-03-26 §
23:07 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: ce63a4e: Enable wmgUseFooterContactLink for cswiki (T248584; take II) (duration: 00m 57s) [production]
23:05 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: ce63a4e: Enable wmgUseFooterContactLink for cswiki (T248584) (duration: 00m 58s) [production]
22:51 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.25/includes/user/UserRightsProxy.php: I9121f5aae (4/4) (duration: 00m 58s) [production]
22:50 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.25/includes/search/SearchMySQL.php: I9121f5aae (3/4) (duration: 00m 58s) [production]
22:48 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.25/includes/objectcache/SqlBagOStuff.php: I9121f5aae (2/4) (duration: 00m 58s) [production]
22:44 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.25/includes/jobqueue/jobs/RecentChangesUpdateJob.php: I9121f5aae (1/4) (duration: 01m 00s) [production]
22:05 <ejegg> updated fundraising CiviCRM from f1cb23e809 to cf2e2c11c3 [production]
21:43 <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.25/extensions/MachineVision: Fix: Stop sorting label suggestions by Wikidata ID in ApiQueryImageLabels (duration: 01m 00s) [production]
21:34 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
21:32 <cdanis> cdanis@re0.cr1-eqsin# set chassis afeb slot 0 inline-services flex-flow-sizing cdanis@re0.cr1-eqsin# commit comment "flex-flow-sizing T248394" [production]
21:31 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
21:30 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
21:27 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
21:27 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@f34260c]: Update mobileapps to 3f30f20c (duration: 03m 07s) [production]
21:24 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@f34260c]: Update mobileapps to 3f30f20c [production]
21:15 <cdanis> repool ulsfo [production]
21:12 <cdanis> applied flow-table-size configuration to cr4-ulsfo which did not need a reboot to apply it T248394 [production]
16:52 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
16:50 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
16:43 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
16:40 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
16:34 <XioNoX> stop exchanging full BGP view between eqiad and codfw - T246721 [production]
16:19 <pt1979@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:18 <XioNoX> stop advertising 208.80.152.0/22 from eqiad - T246721 [production]
16:15 <mutante> signing puppet cert for miscweb1002, installed buster, added insetup role (T247887) [production]
16:15 <ebernhardson> set cloudelastic-chi wikidatawiki_content to 0 replicas while reindexing [production]
16:14 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
16:14 <moritzm> rebooting mw2150 for some tests [production]
16:12 <XioNoX> stop advertising 2620:0:860::/46 from eqiad - T246721 [production]
16:12 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
16:11 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:11 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
16:10 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:58 <volans@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:53 <volans@cumin1001> START - Cookbook sre.dns.netbox [production]
15:51 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:51 <moritzm> installing grub2 updates from Stretch point release [production]
15:49 <XioNoX> start advertising 208.80.154.0/23 from eqiad - T246721 [production]
15:49 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:48 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:46 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:40 <XioNoX> start advertising 2620:0:861::/48 from eqiad - T246721 [production]
15:20 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:17 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:15 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:12 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]