751-800 of 10000 results (29ms)
2020-07-08 §
07:49 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single [production]
07:48 <jynus> stop bacula-director on backup1001 in preparation for m1 switchover T256717 [production]
07:47 <akosiaris@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:47 <akosiaris@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:47 <akosiaris@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:47 <akosiaris@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:47 <akosiaris@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
07:47 <akosiaris@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:45 <moritzm> installing PHP 7.3 security updates [production]
07:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1143', diff saved to https://phabricator.wikimedia.org/P11802 and previous config saved to /var/cache/conftool/dbconfig/20200708-073548-marostegui.json [production]
07:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1143', diff saved to https://phabricator.wikimedia.org/P11801 and previous config saved to /var/cache/conftool/dbconfig/20200708-073037-marostegui.json [production]
07:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1142', diff saved to https://phabricator.wikimedia.org/P11800 and previous config saved to /var/cache/conftool/dbconfig/20200708-073011-marostegui.json [production]
07:24 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1142', diff saved to https://phabricator.wikimedia.org/P11799 and previous config saved to /var/cache/conftool/dbconfig/20200708-072431-marostegui.json [production]
07:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1141', diff saved to https://phabricator.wikimedia.org/P11798 and previous config saved to /var/cache/conftool/dbconfig/20200708-070921-marostegui.json [production]
07:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1141', diff saved to https://phabricator.wikimedia.org/P11797 and previous config saved to /var/cache/conftool/dbconfig/20200708-070432-marostegui.json [production]
07:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1138', diff saved to https://phabricator.wikimedia.org/P11796 and previous config saved to /var/cache/conftool/dbconfig/20200708-070403-marostegui.json [production]
06:47 <marostegui> start topology changes on m1 T256717 [production]
06:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1138', diff saved to https://phabricator.wikimedia.org/P11795 and previous config saved to /var/cache/conftool/dbconfig/20200708-064354-marostegui.json [production]
06:36 <marostegui> Deploy schema change on s2 primary master db1122 T238966 [production]
06:18 <_joe_> rolling restart of restbase to pick up the proton url change [production]
03:36 <andrew@deploy1001> Finished deploy [horizon/deploy@505819d]: further fixes for proxy editing --bug 610130 (duration: 03m 44s) [production]
03:32 <andrew@deploy1001> Started deploy [horizon/deploy@505819d]: further fixes for proxy editing --bug 610130 [production]
2020-07-07 §
22:41 <mutante> new Wikimedia Annual Report 2019 now available on annual.wikimedia.org [production]
21:29 <andrew@deploy1001> Finished deploy [horizon/deploy@fce8183]: further fixes for proxy editing --bug 610130 (duration: 03m 35s) [production]
21:25 <andrew@deploy1001> Started deploy [horizon/deploy@fce8183]: further fixes for proxy editing --bug 610130 [production]
21:10 <andrew@deploy1001> Finished deploy [horizon/deploy@abcd051]: further fixes for proxy editing --bug 610130 (duration: 03m 26s) [production]
21:07 <andrew@deploy1001> Started deploy [horizon/deploy@abcd051]: further fixes for proxy editing --bug 610130 [production]
20:41 <ppchelko@deploy1001> Finished deploy [restbase/deploy@05b8bd5]: Remove restbase2009, take 2 (duration: 09m 15s) [production]
20:32 <ppchelko@deploy1001> Started deploy [restbase/deploy@05b8bd5]: Remove restbase2009, take 2 [production]
20:32 <ppchelko@deploy1001> Finished deploy [restbase/deploy@05b8bd5]: Remove restbase2009 (duration: 14m 28s) [production]
20:24 <mutante> kubernetes1003 - starting nagios-nrpe-server [production]
20:23 <mutante> kubernetes1001 - starting nagios-nrpe-server [production]
20:17 <ppchelko@deploy1001> Started deploy [restbase/deploy@05b8bd5]: Remove restbase2009 [production]
19:29 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
19:27 <mutante> destroying VM gerrit1002 - decom cookbook [production]
19:26 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
19:18 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.35.0-wmf.40 refs T256668 [production]
19:04 <mutante> contint2001 - move /var/lib/zuul/.ssh/known_hosts to root and run puppet to recreate it [production]
18:38 <andrew@deploy1001> Finished deploy [horizon/deploy@eaa056e]: fix for proxy editing --bug 610130 (duration: 03m 18s) [production]
18:35 <andrew@deploy1001> Started deploy [horizon/deploy@eaa056e]: fix for proxy editing --bug 610130 [production]
18:27 <andrew@deploy1001> Finished deploy [horizon/deploy@a39e86c]: update proxy UI to support editing existing proxies (duration: 03m 26s) [production]
18:23 <andrew@deploy1001> Started deploy [horizon/deploy@a39e86c]: update proxy UI to support editing existing proxies [production]
18:10 <krinkle@deploy1001> Synchronized w/: remove untracked test cookie file (duration: 01m 04s) [production]
18:08 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.40/includes/Revision/RevisionStore.php: I8f986daeab4 (duration: 01m 05s) [production]
17:59 <herron> imported (logstash|kibana|elasticsearch)-oss-7.8.0 into buster-wikimedia thirdparty/elastic78 [production]
17:54 <hnowlan> finished removing restbase2009 from cassandra pool [production]
17:06 <hnowlan> removed restbase2009-b from cassandra pool, removing restbase2009-c [production]
16:40 <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.35.0-wmf.40/extensions/Wikibase: Backport: [[gerrit:610086|Revert "Don’t load $wgWBClientSettings in WikibaseClient.php" (T257296)]] (duration: 01m 10s) [production]
15:49 <hnowlan> running nodetool removenode for restbase2009-a [production]
15:38 <hnowlan@deploy1001> Started restart [restbase/deploy@05b8bd5]: Restarting restbase after removal of restbase2009 [production]