51-100 of 10000 results (12ms)
2020-07-07 ยง
14:30 <otto@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'eventgate-analytics' for release 'production' . [production]
14:24 <otto@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'production' . [production]
14:24 <otto@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'canary' . [production]
14:20 <otto@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'canary' . [production]
14:20 <otto@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'production' . [production]
14:16 <hashar@deploy1001> Finished deploy [integration/docroot@708d3eb]: (no justification provided) (duration: 00m 09s) [production]
14:16 <hashar@deploy1001> Started deploy [integration/docroot@708d3eb]: (no justification provided) [production]
13:38 <_joe_> rolling restart of restbase to pick up using envoy [production]
13:31 <otto@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'canary' . [production]
13:31 <otto@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'production' . [production]
13:29 <XioNoX> cr2-eqiad> request vmhost snapshot routing-engine both - T257153 [production]
13:24 <XioNoX> cr1-eqiad> request vmhost snapshot routing-engine both - T257153 [production]
13:15 <kormat@cumin1001> dbctl commit (dc=all): 'Promote es2021 to es4 master T257284', diff saved to https://phabricator.wikimedia.org/P11789 and previous config saved to /var/cache/conftool/dbconfig/20200707-131524-kormat.json [production]
12:59 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:57 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:44 <kormat> starting (codfw) es5 failover from es2020 to es2021 T257284 [production]
12:30 <kormat@cumin1001> dbctl commit (dc=all): 'Set es2021 to weight 50 T257284', diff saved to https://phabricator.wikimedia.org/P11787 and previous config saved to /var/cache/conftool/dbconfig/20200707-123003-kormat.json [production]
12:12 <jforrester@deploy1001> Finished scap: Full scap and testwikis to 1.35.0-wmf.40 for T256668 (duration: 33m 09s) [production]
12:01 <marostegui> Deploy schema change on labswiki (wikitech) master - T253276 [production]
11:58 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1082', diff saved to https://phabricator.wikimedia.org/P11786 and previous config saved to /var/cache/conftool/dbconfig/20200707-115838-marostegui.json [production]
11:39 <jforrester@deploy1001> Started scap: Full scap and testwikis to 1.35.0-wmf.40 for T256668 [production]
11:38 <jforrester@deploy1001> scap failed: LockFailedError Failed to acquire lock "/var/lock/scap.operations_mediawiki-config.lock"; owner is "jforrester"; reason is "testwikis wikis to 1.35.0-wmf.40" (duration: 00m 00s) [production]
11:33 <moritzm> installing PHP 7.0 security updates [production]
11:29 <marostegui> Deploy schema change on db1082, this will create lag on s5 labs [production]
11:29 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1082', diff saved to https://phabricator.wikimedia.org/P11784 and previous config saved to /var/cache/conftool/dbconfig/20200707-112926-marostegui.json [production]
11:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1130', diff saved to https://phabricator.wikimedia.org/P11783 and previous config saved to /var/cache/conftool/dbconfig/20200707-112830-marostegui.json [production]
11:26 <godog> test bumping logstash7 batch size to 256 [production]
11:17 <moritzm> prune PHP 7.0 packages from mwdebug1001/2001/2002 [production]
11:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1130', diff saved to https://phabricator.wikimedia.org/P11782 and previous config saved to /var/cache/conftool/dbconfig/20200707-110506-marostegui.json [production]
11:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1110', diff saved to https://phabricator.wikimedia.org/P11781 and previous config saved to /var/cache/conftool/dbconfig/20200707-110412-marostegui.json [production]
10:57 <moritzm> prune PHP 7.0 packages from mw2190-mw2214 [production]
10:46 <jforrester@deploy1001> Started scap: testwikis wikis to 1.35.0-wmf.40 [production]
10:44 <jforrester@deploy1001> Pruned MediaWiki: 1.35.0-wmf.38 (duration: 17m 23s) [production]
10:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1110', diff saved to https://phabricator.wikimedia.org/P11780 and previous config saved to /var/cache/conftool/dbconfig/20200707-103255-marostegui.json [production]
10:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1113:3315', diff saved to https://phabricator.wikimedia.org/P11779 and previous config saved to /var/cache/conftool/dbconfig/20200707-102757-marostegui.json [production]
10:26 <moritzm> prune PHP 7.0 packages from mw2135-mw2147 [production]
10:12 <addshore@deploy1001> Synchronized wmf-config/config/testcommonswiki.yaml: [[gerrit:609985]] Make testcommonswiki a testwikidata client T257266 PT2/2 (duration: 00m 55s) [production]
10:11 <addshore@deploy1001> sync-file aborted: [[gerrit:609985]] Make testcommonswiki a testwikidata client T257266 PT1/2 (duration: 00m 00s) [production]
10:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1113:3315', diff saved to https://phabricator.wikimedia.org/P11778 and previous config saved to /var/cache/conftool/dbconfig/20200707-101043-marostegui.json [production]
10:10 <addshore@deploy1001> Synchronized dblists/wikidataclient-test.dblist: [[gerrit:609985]] Make testcommonswiki a testwikidata client T257266 PT1/2 (duration: 00m 56s) [production]
10:08 <addshore@deploy1001> sync-file aborted: [[gerrit:609985]] Make testcommonswiki a testwikidata client T257266 PT1/2 (duration: 00m 36s) [production]
10:06 <elukey> decommission archiva1001 [production]
10:05 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
10:03 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1144:3315', diff saved to https://phabricator.wikimedia.org/P11777 and previous config saved to /var/cache/conftool/dbconfig/20200707-100328-marostegui.json [production]
10:03 <elukey@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
10:03 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:03 <elukey@cumin1001> START - Cookbook sre.hosts.decommission [production]
09:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1144:3315', diff saved to https://phabricator.wikimedia.org/P11776 and previous config saved to /var/cache/conftool/dbconfig/20200707-095443-marostegui.json [production]
09:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1096:3315', diff saved to https://phabricator.wikimedia.org/P11775 and previous config saved to /var/cache/conftool/dbconfig/20200707-095428-marostegui.json [production]
09:42 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:609971]] T257266 Enable sitelinks to testcommons from test wikidata sites (duration: 00m 56s) [production]