701-750 of 10000 results (22ms)
2020-09-23 ยง
15:45 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:44 <elukey@cumin1001> END (FAIL) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=99) [production]
15:44 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:43 <elukey@cumin1001> END (FAIL) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=99) [production]
15:43 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:43 <kormat@cumin1001> dbctl commit (dc=all): 'db2126 (re)pooling @ 75%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12773 and previous config saved to /var/cache/conftool/dbconfig/20200923-154315-kormat.json [production]
15:40 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) [production]
15:37 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:33 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) [production]
15:30 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:28 <kormat@cumin1001> dbctl commit (dc=all): 'db2126 (re)pooling @ 50%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12772 and previous config saved to /var/cache/conftool/dbconfig/20200923-152812-kormat.json [production]
15:21 <elukey@cumin1001> END (FAIL) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=99) [production]
15:21 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:13 <kormat@cumin1001> dbctl commit (dc=all): 'db2126 (re)pooling @ 25%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12771 and previous config saved to /var/cache/conftool/dbconfig/20200923-151308-kormat.json [production]
14:48 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:48 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:44 <kormat@cumin1001> dbctl commit (dc=all): 'db2126 depooling: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12770 and previous config saved to /var/cache/conftool/dbconfig/20200923-144441-kormat.json [production]
14:44 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:44 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:37 <herron> grew prometheus1004 prometheus-ops filesystem to 1.6T [production]
14:35 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:629383|Enable repo config propagateChangeVisibility everywhere]], 2/2 (duration: 01m 06s) [production]
14:33 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:629383|Enable repo config propagateChangeVisibility everywhere]], 1/2 (duration: 01m 06s) [production]
13:50 <kormat@cumin1001> dbctl commit (dc=all): 'db2104 (re)pooling @ 100%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12769 and previous config saved to /var/cache/conftool/dbconfig/20200923-135028-kormat.json [production]
13:35 <kormat@cumin1001> dbctl commit (dc=all): 'db2104 (re)pooling @ 75%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12768 and previous config saved to /var/cache/conftool/dbconfig/20200923-133525-kormat.json [production]
13:29 <marostegui@cumin1001> dbctl commit (dc=all): 'db2074 (re)pooling @ 100%: Slowly repool db2074 ', diff saved to https://phabricator.wikimedia.org/P12766 and previous config saved to /var/cache/conftool/dbconfig/20200923-132918-root.json [production]
13:20 <kormat@cumin1001> dbctl commit (dc=all): 'db2104 (re)pooling @ 50%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12765 and previous config saved to /var/cache/conftool/dbconfig/20200923-132022-kormat.json [production]
13:20 <moritzm> installing ruby-json security updates [production]
13:14 <marostegui@cumin1001> dbctl commit (dc=all): 'db2074 (re)pooling @ 75%: Slowly repool db2074 ', diff saved to https://phabricator.wikimedia.org/P12764 and previous config saved to /var/cache/conftool/dbconfig/20200923-131414-root.json [production]
13:05 <kormat@cumin1001> dbctl commit (dc=all): 'db2104 (re)pooling @ 25%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12763 and previous config saved to /var/cache/conftool/dbconfig/20200923-130518-kormat.json [production]
13:04 <moritzm> installing multipath-tools bugfix updates from buster 10.5 point release [production]
12:59 <marostegui@cumin1001> dbctl commit (dc=all): 'db2074 (re)pooling @ 25%: Slowly repool db2074 ', diff saved to https://phabricator.wikimedia.org/P12762 and previous config saved to /var/cache/conftool/dbconfig/20200923-125911-root.json [production]
12:49 <moritzm> installing libunwind bugfix updates from buster 10.5 point release [production]
12:39 <kormat@cumin1001> dbctl commit (dc=all): 'db2104 depooling: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12761 and previous config saved to /var/cache/conftool/dbconfig/20200923-123922-kormat.json [production]
12:38 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2074', diff saved to https://phabricator.wikimedia.org/P12760 and previous config saved to /var/cache/conftool/dbconfig/20200923-123806-marostegui.json [production]
12:37 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:37 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:36 <kormat@cumin1001> dbctl commit (dc=all): 'Add db2088:3312 to api while db2104 gets depooled T259831', diff saved to https://phabricator.wikimedia.org/P12759 and previous config saved to /var/cache/conftool/dbconfig/20200923-123649-kormat.json [production]
12:35 <marostegui@cumin1001> dbctl commit (dc=all): 'db2074 (re)pooling @ 25%: Slowly db2074 ', diff saved to https://phabricator.wikimedia.org/P12758 and previous config saved to /var/cache/conftool/dbconfig/20200923-123528-root.json [production]
12:22 <ema> cp4027: repool with varnish 6.0.6-1wm1 T263557 [production]
12:09 <ema> cp4027: depool and upgrade varnish to 6.0.6-1wm1 T263557 [production]
11:52 <moritzm> installing GNUTLS bugfix updates from buster 10.5 point release [production]
11:51 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.10/extensions/GrowthExperiments/modules/homepage/suggestededits/ext.growthExperiments.Homepage.GrowthTasksApi.js: 73b5ce82b3913232b708405147f0bb6d27128974: Fix GrowthTasksApi lazy-loading flags for pages with no views (T263611) (duration: 01m 05s) [production]
11:49 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.10/extensions/GrowthExperiments/modules/help/ext.growthExperiments.PostEdit.js: 1ab31a966edc4748f82f75bb370371733c2ca090: Mark pageviews as not used in the mobile postedit notice (T263611) (duration: 01m 06s) [production]
11:38 <Urbanecm> Revert https://gerrit.wikimedia.org/r/c/mediawiki/core/+/629188 and fetch to deploy1001 to unblock EU B&C deployment (T237467; cc twentyafterfour) [production]
11:27 <kormat@cumin1001> dbctl commit (dc=all): 'db2088:3312 (re)pooling @ 100%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12756 and previous config saved to /var/cache/conftool/dbconfig/20200923-112712-kormat.json [production]
11:12 <kormat@cumin1001> dbctl commit (dc=all): 'db2088:3312 (re)pooling @ 75%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12755 and previous config saved to /var/cache/conftool/dbconfig/20200923-111209-kormat.json [production]
11:08 <Urbanecm> Create ContentTranslation tables at testwiki using SQL files from `/srv/mediawiki/php-1.36.0-wmf.10/extensions/ContentTranslation/sql` (T263417 [production]
10:57 <kormat@cumin1001> dbctl commit (dc=all): 'db2088:3312 (re)pooling @ 50%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12754 and previous config saved to /var/cache/conftool/dbconfig/20200923-105705-kormat.json [production]
10:42 <kormat@cumin1001> dbctl commit (dc=all): 'db2088:3312 (re)pooling @ 25%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12753 and previous config saved to /var/cache/conftool/dbconfig/20200923-104202-kormat.json [production]
10:21 <kormat@cumin1001> dbctl commit (dc=all): 'db2088:3312 depooling: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12752 and previous config saved to /var/cache/conftool/dbconfig/20200923-102120-kormat.json [production]