651-700 of 10000 results (61ms)
2020-02-07 ยง
14:43 <hoo@deploy1001> Synchronized wmf-config/Wikibase.php: REVERT: Wikibase Client: Fix setting name typo (T244529) (duration: 01m 40s) [production]
14:43 <Amir1> ladsgroup@mwmaint1002:~$ mwscript createAndPromote.php --wiki=zhwiki --force "Amir Sarabadani (WMDE)" --sysop (T244578) [production]
14:40 <hoo@deploy1001> Scap failed!: 9/11 canaries failed their endpoint checks(http://en.wikipedia.org) [production]
14:38 <hoo@deploy1001> Synchronized wmf-config/Wikibase.php: Wikibase Client: Fix setting name typo (T244529) (duration: 01m 20s) [production]
14:33 <vgutierrez> depool and reimage ncredir3001 as buster - T243391 [production]
14:32 <vgutierrez> depool & reimage cp4031 as buster - T242093 [production]
14:23 <vgutierrez> pooling ncredir3002 running buster - T243391 [production]
13:26 <vgutierrez> pooling cp4021 with buster - T242093 [production]
13:05 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:03 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:51 <vgutierrez> depool and reimage ncredir3002 as buster - T243391 [production]
12:42 <vgutierrez> depool & reimage cp4021 as buster - T242093 [production]
12:08 <akosiaris@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:08 <akosiaris@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:58 <akosiaris@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:57 <akosiaris@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:25 <vgutierrez> pooling ncredir5001 running buster - T243391 [production]
11:24 <vgutierrez> pooling cp4022 with buster - T242093 [production]
11:09 <akosiaris> undo wikifeeds experiments [production]
11:07 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . [production]
10:42 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . [production]
10:40 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:37 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:36 <akosiaris> conduct experiments with stopping/starting uwsgi-ores on ores2001 T242705 [production]
10:24 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . [production]
10:23 <vgutierrez> depool and reimage ncredir5001 as buster - T243391 [production]
10:14 <vgutierrez> depool & reimage cp4022 as buster - T242093 [production]
10:02 <akosiaris> increase capacity for wikifeeds by 50% T244535 [production]
10:01 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . [production]
10:01 <akosiaris@deploy1001> helmfile [CODFW] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . [production]
09:53 <ema> A:mw: increase keepalive_requests from 100 to 200 https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/570670/ T241145 [production]
09:09 <godog> roll restart cassandra instance on restbase-dev [production]
09:03 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . [production]
09:03 <godog> restart cassandra on restbase-dev1004 to test logging pipeline onboard [production]
09:01 <akosiaris@deploy1001> helmfile [CODFW] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . [production]
08:59 <akosiaris@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'wikifeeds' for release 'staging' . [production]
08:58 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1090:3312, db1090:3317', diff saved to https://phabricator.wikimedia.org/P10343 and previous config saved to /var/cache/conftool/dbconfig/20200207-085846-marostegui.json [production]
08:54 <marostegui> Upgrade db1090:3312, db1090:3317 [production]
08:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1090:3312, db1090:3317 for upgrade', diff saved to https://phabricator.wikimedia.org/P10342 and previous config saved to /var/cache/conftool/dbconfig/20200207-085432-marostegui.json [production]
08:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1101:3317 T239453', diff saved to https://phabricator.wikimedia.org/P10341 and previous config saved to /var/cache/conftool/dbconfig/20200207-084447-marostegui.json [production]
08:44 <moritzm> installing libexif security updates [production]
08:21 <akosiaris> deploy https://gerrit.wikimedia.org/r/570726 T244535 to avoid CPU throttling of wikifeeds [production]
08:21 <akosiaris@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'wikifeeds' for release 'staging' . [production]
07:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Increase base weight for db1126', diff saved to https://phabricator.wikimedia.org/P10340 and previous config saved to /var/cache/conftool/dbconfig/20200207-075323-marostegui.json [production]
07:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1101:3317 T239453', diff saved to https://phabricator.wikimedia.org/P10339 and previous config saved to /var/cache/conftool/dbconfig/20200207-075234-marostegui.json [production]
07:48 <marostegui> Remove revision partitions from db2085:3318 T239453 [production]
07:45 <marostegui@cumin1001> dbctl commit (dc=all): 'Fullyy repool db1126 T232446', diff saved to https://phabricator.wikimedia.org/P10338 and previous config saved to /var/cache/conftool/dbconfig/20200207-074511-marostegui.json [production]
07:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2085:3318 T239453', diff saved to https://phabricator.wikimedia.org/P10337 and previous config saved to /var/cache/conftool/dbconfig/20200207-074407-marostegui.json [production]
07:42 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1101:3317 T239453', diff saved to https://phabricator.wikimedia.org/P10336 and previous config saved to /var/cache/conftool/dbconfig/20200207-074258-marostegui.json [production]
07:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1101:3317 T239453', diff saved to https://phabricator.wikimedia.org/P10335 and previous config saved to /var/cache/conftool/dbconfig/20200207-073130-marostegui.json [production]