201-250 of 10000 results (17ms)
2020-09-23 ยง
18:54 <ryankemper> `scap sync-file wmf-config/ProductionServices.php 'Config: [[gerrit:628978|cloudelastic: envoy sits in front now (T263073)]]'` from `ryankemper@deploy1001:/srv/mediawiki-staging` [production]
18:47 <ryankemper> Above deploy appears successful, test requests seem to be taking 40ms instead of the previous 140ms [production]
18:31 <ryankemper> HEAD of `/srv/mediawiki-staging` is now at 7a96d63d862eacf5244eec79b63d29d78fbaa6f7 as expected [production]
18:13 <Urbanecm> End of [urbanecm@mwmaint2001 ~]$ mwscript updateCollation.php --wiki=trwikiquote --previous-collation=uppercase # T263628 [production]
18:13 <Urbanecm> Start of [urbanecm@mwmaint2001 ~]$ mwscript updateCollation.php --wiki=trwikiquote --previous-collation=uppercase # T263628 [production]
18:12 <Urbanecm> urbanecm@deploy1001: scap sync-file wmf-config/InitialiseSettings.php 'b1554f36be68106c9364f4aa2fd70d759ad74356: Set $wgCategoryCollation = uca-tr on trwikiquote (T263628)' [production]
18:11 <Urbanecm> Logmsgbot seems to be down [production]
17:29 <robh> migrating ganeti instances off ganeti5002 for troubleshooting per T261130 [production]
16:37 <sukhe> upload dnsdist_1.4.0-1~deb10u2 to apt.wm.o (buster) - T252132 [production]
16:00 <herron> switching icinga over from icinga1001 to alert1001 T247966 [production]
16:00 <kormat@cumin1001> dbctl commit (dc=all): 'Remove db2088:3312 from api now that db2104/db2126 are done T259831', diff saved to https://phabricator.wikimedia.org/P12775 and previous config saved to /var/cache/conftool/dbconfig/20200923-160010-kormat.json [production]
15:58 <kormat@cumin1001> dbctl commit (dc=all): 'db2126 (re)pooling @ 100%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12774 and previous config saved to /var/cache/conftool/dbconfig/20200923-155819-kormat.json [production]
15:57 <robh> updating firmware on mw1360, troubleshooting nic failure issue T262151 [production]
15:57 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.10/includes/specials/SpecialBlock.php: 3234fad0d9b370b1cf75093dd13c0e1639619f08: SpecialUnblock: Allow getTargetAndType to accept null $par (T263642) (duration: 01m 07s) [production]
15:56 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.10/includes/specials/SpecialUnblock.php: 3234fad0d9b370b1cf75093dd13c0e1639619f08: SpecialUnblock: Allow getTargetAndType to accept null $par (T263642) (duration: 01m 08s) [production]
15:53 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:52 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) [production]
15:51 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
15:50 <pt1979@cumin2001> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]
15:48 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:48 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) [production]
15:45 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
15:45 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:44 <elukey@cumin1001> END (FAIL) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=99) [production]
15:44 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:43 <elukey@cumin1001> END (FAIL) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=99) [production]
15:43 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:43 <kormat@cumin1001> dbctl commit (dc=all): 'db2126 (re)pooling @ 75%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12773 and previous config saved to /var/cache/conftool/dbconfig/20200923-154315-kormat.json [production]
15:40 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) [production]
15:37 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:33 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) [production]
15:30 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:28 <kormat@cumin1001> dbctl commit (dc=all): 'db2126 (re)pooling @ 50%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12772 and previous config saved to /var/cache/conftool/dbconfig/20200923-152812-kormat.json [production]
15:21 <elukey@cumin1001> END (FAIL) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=99) [production]
15:21 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers [production]
15:13 <kormat@cumin1001> dbctl commit (dc=all): 'db2126 (re)pooling @ 25%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12771 and previous config saved to /var/cache/conftool/dbconfig/20200923-151308-kormat.json [production]
14:48 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:48 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:44 <kormat@cumin1001> dbctl commit (dc=all): 'db2126 depooling: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12770 and previous config saved to /var/cache/conftool/dbconfig/20200923-144441-kormat.json [production]
14:44 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:44 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:37 <herron> grew prometheus1004 prometheus-ops filesystem to 1.6T [production]
14:35 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:629383|Enable repo config propagateChangeVisibility everywhere]], 2/2 (duration: 01m 06s) [production]
14:33 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:629383|Enable repo config propagateChangeVisibility everywhere]], 1/2 (duration: 01m 06s) [production]
13:50 <kormat@cumin1001> dbctl commit (dc=all): 'db2104 (re)pooling @ 100%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12769 and previous config saved to /var/cache/conftool/dbconfig/20200923-135028-kormat.json [production]
13:35 <kormat@cumin1001> dbctl commit (dc=all): 'db2104 (re)pooling @ 75%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12768 and previous config saved to /var/cache/conftool/dbconfig/20200923-133525-kormat.json [production]
13:29 <marostegui@cumin1001> dbctl commit (dc=all): 'db2074 (re)pooling @ 100%: Slowly repool db2074 ', diff saved to https://phabricator.wikimedia.org/P12766 and previous config saved to /var/cache/conftool/dbconfig/20200923-132918-root.json [production]
13:20 <kormat@cumin1001> dbctl commit (dc=all): 'db2104 (re)pooling @ 50%: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12765 and previous config saved to /var/cache/conftool/dbconfig/20200923-132022-kormat.json [production]
13:20 <moritzm> installing ruby-json security updates [production]
13:14 <marostegui@cumin1001> dbctl commit (dc=all): 'db2074 (re)pooling @ 75%: Slowly repool db2074 ', diff saved to https://phabricator.wikimedia.org/P12764 and previous config saved to /var/cache/conftool/dbconfig/20200923-131414-root.json [production]