1901-1950 of 10000 results (70ms)
2019-12-19 ยง
16:10 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:53 <XioNoX> enable netflow sampling in ulsfo [production]
15:34 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:34 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:12 <ejegg> updated fundraising internal dashboard from 913d690621 to b75f9074de [production]
15:10 <ejegg> updated payments-wiki from 7131303dba to 827e3235dc [production]
14:50 <reedy@deploy1001> Synchronized wmf-config/CommonSettings.php: Mark REL1_34 stable in ExtensionDistributor (duration: 00m 53s) [production]
14:47 <_joe_> restart pybal on the primary low-traffic balancers in eqiad, codfw [production]
14:41 <ema> cp1075, cp4028: ats-backend-restart to disable xdebug plugin T241001 [production]
14:40 <_joe_> restarting pybal on the backup low-traffic in eqiad,codfw [production]
14:12 <phamhi@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:10 <phamhi@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:48 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:45 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:41 <phamhi@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:41 <phamhi@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1136', diff saved to https://phabricator.wikimedia.org/P9979 and previous config saved to /var/cache/conftool/dbconfig/20191219-133525-marostegui.json [production]
13:34 <ema> pool cp1089 with ATS backend T227432 [production]
13:33 <ema> pool cp2023 with ATS backend T227432 [production]
13:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1136', diff saved to https://phabricator.wikimedia.org/P9978 and previous config saved to /var/cache/conftool/dbconfig/20191219-131832-marostegui.json [production]
13:17 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:15 <James_F> mwscript emptyUserGroup.php --wiki=mediawikiwiki oauthadmin T241142 [production]
13:15 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:12 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:12 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1136', diff saved to https://phabricator.wikimedia.org/P9977 and previous config saved to /var/cache/conftool/dbconfig/20191219-130832-marostegui.json [production]
13:02 <jforrester@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.11 [production]
12:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1136', diff saved to https://phabricator.wikimedia.org/P9976 and previous config saved to /var/cache/conftool/dbconfig/20191219-125748-marostegui.json [production]
12:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1136, s7 candidate master, for upgrade', diff saved to https://phabricator.wikimedia.org/P9975 and previous config saved to /var/cache/conftool/dbconfig/20191219-125314-marostegui.json [production]
12:52 <moritzm> failover ganeti master in ulsfo to ganeti4002 for a test [production]
12:52 <jforrester@deploy1001> Synchronized php-1.35.0-wmf.11/extensions/WikiLove/includes/ApiWikiLove.php: T241094 ApiWikiLove: Don't pass null to implode(), but fall back to [] (duration: 01m 02s) [production]
12:52 <ema> depool cp2023 and cp1089 for ATS reimages T227432. Reimaged together because of T238817 [production]
12:37 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: Cleanup: Remove CLI 'display_errors=stderr' setting (duration: 01m 01s) [production]
12:37 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:35 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:32 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: Cleanup: Remove very old 'error_append_string' INI override (duration: 01m 02s) [production]
12:30 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: Cleanup: Move core DB/SQL-related config closer together (duration: 01m 02s) [production]
12:03 <moritzm> installing netflow4001 [production]
11:43 <addshore> addshore@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=wikidatawiki --batch-size=5 --from-id=4089887 # T237984 - Will stop after P433 [production]
11:28 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:26 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:05 <moritzm> removing kubestagetcd1001-1003 from debmonitor T224568 [production]
11:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1104', diff saved to https://phabricator.wikimedia.org/P9974 and previous config saved to /var/cache/conftool/dbconfig/20191219-110404-marostegui.json [production]
10:50 <ema> pool cp3064 with ATS backend T227432 [production]
10:40 <effie> pool mw1320 [production]
10:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1104', diff saved to https://phabricator.wikimedia.org/P9973 and previous config saved to /var/cache/conftool/dbconfig/20191219-102316-marostegui.json [production]
10:21 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:18 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1104', diff saved to https://phabricator.wikimedia.org/P9972 and previous config saved to /var/cache/conftool/dbconfig/20191219-101024-marostegui.json [production]
10:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1093', diff saved to https://phabricator.wikimedia.org/P9971 and previous config saved to /var/cache/conftool/dbconfig/20191219-100938-marostegui.json [production]