101-150 of 10000 results (16ms)
2020-06-30 §
10:37 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:30 <hashar@deploy1001> Synchronized php-1.35.0-wmf.39/includes/specials/SpecialUndelete.php: Remove another use of PageArchive::getRevision - T249982 T254176 (duration: 00m 56s) [production]
10:09 <marostegui> Deploy schema change on db1076 [production]
10:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1076', diff saved to https://phabricator.wikimedia.org/P11708 and previous config saved to /var/cache/conftool/dbconfig/20200630-100912-marostegui.json [production]
10:04 <vgutierrez> rolling restart of eqiad cache nodes to catch up on kernel upgrades [production]
10:03 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:03 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:02 <volker-e@deploy1001> Finished deploy [design/style-guide@e3fda83]: Deploy design/style-guide: (duration: 00m 07s) [production]
10:02 <volker-e@deploy1001> Started deploy [design/style-guide@e3fda83]: Deploy design/style-guide: [production]
09:47 <hashar@deploy1001> Pruned MediaWiki: 1.35.0-wmf.37 (duration: 02m 20s) [production]
09:40 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:40 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:21 <hashar@deploy1001> Pruned MediaWiki: 1.35.0-wmf.36 (duration: 28m 11s) [production]
08:54 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:53 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:53 <hashar@deploy1001> clean aborted: Pruned MediaWiki: 1.35.0-wmf.36 (duration: 00m 00s) [production]
08:51 <hashar> Applied security patches to wmf/1.35.0-wmf.39 # T254176 [production]
08:51 <vgutierrez> rolling restart of codfw cp nodes after "re-formatting" nvme devices - T256655 [production]
08:23 <vgutierrez> repool cp3053 - T256632 [production]
08:10 <hashar> 1.35.0-wmf.39 was branched at e169e3dabcb2217809fc41ba44b43a39ae1a678e T254176 [production]
08:05 <marostegui> Stop MySQL on db1117:3322 to clone db1080 (this will trigger haproxy alerts) - T256717 [production]
08:05 <vgutierrez> powercycle cp3053 (unresponsive after reboot) - T256632 [production]
08:01 <jbond42> disable puppet to restart puppetmasters front ends [production]
07:42 <vgutierrez> reboot cp3053 - T256632 [production]
05:51 <jhuneidi@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . [production]
05:13 <marostegui> Deploy schema change on s8 codfw - T256680 [production]
04:57 <marostegui> remove pl_from index from db1141, db1121, db1148 - T256684 [production]
04:57 <jhuneidi@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . [production]
04:56 <marostegui> Remove plfrom from db1096:3316 and db1098:3316 - T256684 [production]
2020-06-29 §
23:28 <eileen> civicrm revision changed from 52a32f2d66 to 391d0fdf75, config revision is f1b4bdb7b7 [production]
22:00 <sbassett> Deployed patch for T256171 [production]
21:56 <sbassett> Deployed patch for T255918 [production]
20:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1144:3315 T256679', diff saved to https://phabricator.wikimedia.org/P11699 and previous config saved to /var/cache/conftool/dbconfig/20200629-200002-marostegui.json [production]
19:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1144:3315 T256679', diff saved to https://phabricator.wikimedia.org/P11698 and previous config saved to /var/cache/conftool/dbconfig/20200629-194327-marostegui.json [production]
18:55 <shdubsh> test mtail rc35+wmf2 on cp5001 - T255776 [production]
18:15 <Urbanecm> Morning B&C done [production]
18:15 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: c86fcd4: Add HTTP proxy to MediaModeration (T247943) (duration: 00m 58s) [production]
18:10 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: aeb7b52: Setup rollbacker and mover on lijwiki (T256109) (duration: 02m 05s) [production]
17:30 <sukhe> LDAP - added datn to groups wmde, nda - T254442 [production]
15:43 <akosiaris@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:43 <akosiaris@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:37 <akosiaris@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:37 <akosiaris@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1105:3312', diff saved to https://phabricator.wikimedia.org/P11696 and previous config saved to /var/cache/conftool/dbconfig/20200629-153140-marostegui.json [production]
15:20 <gehel> repool wdqs1004 - catched up on lag [production]
14:50 <hnowlan@deploy1001> Finished deploy [restbase/deploy@900bcf6]: Redeploy to fix transient error in gom wiktionary deploy (duration: 00m 06s) [production]
14:50 <hnowlan@deploy1001> Started deploy [restbase/deploy@900bcf6]: Redeploy to fix transient error in gom wiktionary deploy [production]
14:48 <hnowlan@deploy1001> Finished deploy [restbase/deploy@900bcf6]: Enable gom wiktionary (duration: 13m 40s) [production]
14:34 <hnowlan@deploy1001> Started deploy [restbase/deploy@900bcf6]: Enable gom wiktionary [production]
14:33 <hnowlan@deploy1001> Finished deploy [restbase/deploy@900bcf6]: Enable gom wiktionary (duration: 17m 49s) [production]