601-650 of 10000 results (23ms)
2020-09-30 ยง
17:52 <bblack> lvs1016: restart pybal [production]
17:04 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:02 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:01 <hnowlan> finished adding restbase2018-a to the cassandra cluster [production]
16:37 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:33 <cicalese@deploy1001> Synchronized wmf-config/CommonSettings-labs.php: Add beta config for API Portal/OAuth communications (duration: 00m 58s) [production]
16:31 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
16:21 <mutante> re-enabled puppet on install2003 [production]
16:21 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:20 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:20 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:20 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:28 <moritzm> removed librsvg 2.40.20-3+wmf1+stretch1 from component/thumbor, superseded by 2.40.21-0+deb9u1 released via stretch-security [production]
14:23 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:22 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:22 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:22 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:22 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:22 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:20 <hnowlan@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
14:20 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:20 <hnowlan@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
14:20 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:10 <cmjohnson1> powering down ores100[3-9 to upgrade memory in each T259909 [production]
14:05 <elukey> create thirdparty/amd-rocm33 for stretch-wikimedia [production]
14:03 <cmjohnson1> powering down ores1002 to upgrade memory T259909 [production]
13:55 <cmjohnson1> powering down ores1001 to upgrade memory T259909 [production]
13:27 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:27 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:27 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:27 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:12 <hnowlan> started bootstrapping restbase1028-a, first buster restbase host [production]
12:39 <marostegui> Deploy schema change on db2080, db2081 T264109 [production]
12:38 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2081', diff saved to https://phabricator.wikimedia.org/P12858 and previous config saved to /var/cache/conftool/dbconfig/20200930-123851-marostegui.json [production]
12:38 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2081', diff saved to https://phabricator.wikimedia.org/P12857 and previous config saved to /var/cache/conftool/dbconfig/20200930-123824-marostegui.json [production]
12:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db2080', diff saved to https://phabricator.wikimedia.org/P12856 and previous config saved to /var/cache/conftool/dbconfig/20200930-123753-marostegui.json [production]
12:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2080', diff saved to https://phabricator.wikimedia.org/P12855 and previous config saved to /var/cache/conftool/dbconfig/20200930-123659-marostegui.json [production]
11:33 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:33 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:33 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:33 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:33 <effie> enable puppet P:mediawiki::mcrouter_wancache for 630845 - T244340 [production]
11:21 <nikerabbit@deploy1001> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:627744|Enable Special:TranslationStats (T263004)]] (duration: 00m 59s) [production]
11:06 <effie> disable puppet on P:mediawiki::mcrouter_wancache for 630845 - T244340 [production]
10:57 <moritzm> installing librsvg security updates [production]
10:47 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
10:47 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
10:44 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
10:44 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
10:34 <hnowlan@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]