2551-2600 of 10000 results (32ms)
2020-08-03 §
11:03 <urbanecm@deploy1001> Synchronized wmf-config/throttle.php: ead6b9eb699594583b06b8f5c23d40d9add2eb49: New throttle rule for Czech editathon (T259352) (duration: 01m 06s) [production]
11:03 <moritzm> installing ruby2.5 security updates [production]
11:01 <moritzm> removing cloudcephmon100[1-3].wikimedia.org from debmonitor (these eventually got re-installed as cloudcephmon100[1-3].eqiad.wmnet) [production]
10:51 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:618020| Bumping portals to master (T128546)]] (duration: 01m 06s) [production]
10:50 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:618020| Bumping portals to master (T128546)]] (duration: 01m 08s) [production]
10:29 <moritzm> installing NSS security updates on buster [production]
10:26 <moritzm> restarting Apache on puppetboard to pick up curl security updates [production]
10:19 <moritzm> restarting wtp1025 (parsoid canary) to pick up curl security updates [production]
09:46 <moritzm> restarting mw1261-mw1265 to pick up curl security updates [production]
09:42 <moritzm> installing curl security updates on stretch [production]
08:59 <moritzm> installing ffmpeg security updates on jobrunners/video scalers (3.2.15 rebuilt with VP9/row-mt patches) [production]
08:26 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1089 into API', diff saved to https://phabricator.wikimedia.org/P12141 and previous config saved to /var/cache/conftool/dbconfig/20200803-082641-marostegui.json [production]
08:25 <moritzm> installing qemu security updates on stretch [production]
08:25 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1106 after compression', diff saved to https://phabricator.wikimedia.org/P12140 and previous config saved to /var/cache/conftool/dbconfig/20200803-082533-marostegui.json [production]
08:22 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Clarify s5 wikis T259437 (duration: 01m 05s) [production]
08:21 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Clarify s5 wikis T259437 (duration: 01m 40s) [production]
08:07 <elukey> roll restart aqs on aqs* to pick up new druid settings [production]
07:10 <marostegui> Remove revision triggers from db2095:3317 for MCR changes T238966 [production]
07:09 <marostegui> Deploy MCR change on s7 codfw, lag will appear on codfw T238966 [production]
07:07 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1106 after compression', diff saved to https://phabricator.wikimedia.org/P12139 and previous config saved to /var/cache/conftool/dbconfig/20200803-070702-marostegui.json [production]
05:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1106 after compression', diff saved to https://phabricator.wikimedia.org/P12138 and previous config saved to /var/cache/conftool/dbconfig/20200803-052715-marostegui.json [production]
05:04 <marostegui> Remove db1108:3321 and db1108:3322 from tendril and add db1108:3351 and db1108:3352 T254462 [production]
05:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1106 after compression', diff saved to https://phabricator.wikimedia.org/P12137 and previous config saved to /var/cache/conftool/dbconfig/20200803-050148-marostegui.json [production]
2020-08-01 §
16:30 <Amir1> wikiadmin@10.64.32.197(avkwiki)> delete from site_identifiers; (T259122) [production]
16:27 <Amir1> start of foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https (T259122) [production]
2020-07-31 §
23:48 <ejegg> updated payments-wiki from c365c136d2 to cd012f37f1 [production]
22:53 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
22:53 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
22:53 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
22:53 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
22:03 <mutante> wtp2019 - parsoid could not start after reimaging - was missing /etc/parsoid/config.yaml which is a symbolic link deep onto /srv/deployment/parsoid/deploy-cache/.. like in some other cases before manually deleted deploy-cache dir and ran puppet again .. T258775 [production]
21:58 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=wtp2020.codfw.wmnet [production]
21:57 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=wtp2019.codfw.wmnet [production]
21:50 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=wtp2019.codfw.wmnet [production]
21:36 <mutante> [wtp2019:~] $ sudo rm -rf /srv/deployment/parsoid/deploy-cache [production]
21:13 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=wtp2018.codfw.wmnet [production]
20:57 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:56 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:56 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:56 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:56 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:56 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:55 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:55 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:55 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:55 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:55 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:55 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:55 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:55 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]