651-700 of 10000 results (27ms)
2021-08-09 §
10:26 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
10:25 <btullis_> btullis@an-druid1003:~$ sudo puppet agent -tv [analytics]
09:55 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
09:50 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
09:49 <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:710925|Increase post edit constraint jobs to 85% of edits (T204031)]] (duration: 00m 58s) [production]
09:46 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on maps1005.eqiad.wmnet with reason: REIMAGE [production]
09:44 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on maps1005.eqiad.wmnet with reason: REIMAGE [production]
09:31 <hnowlan@puppetmaster1001> conftool action : set/pooled=no; selector: name=maps200[1234].codfw.wmnet [production]
08:46 <godog> upgrade prometheus on prometheus2004 - T222113 [production]
08:41 <godog> upgrade prometheus on prometheus1004 - T222113 [production]
08:36 <jynus@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2002.codfw.wmnet with reason: REIMAGE [production]
08:34 <jynus@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2002.codfw.wmnet with reason: REIMAGE [production]
08:24 <marostegui> Upgrade db1117 (all sections) to 10.4.19 [production]
08:03 <ariel@deploy1002> Finished deploy [dumps/dumps@142e91c]: fix for T288192 runnerutils bug (duration: 00m 03s) [production]
08:03 <ariel@deploy1002> Started deploy [dumps/dumps@142e91c]: fix for T288192 runnerutils bug [production]
07:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1160 T288273', diff saved to https://phabricator.wikimedia.org/P16971 and previous config saved to /var/cache/conftool/dbconfig/20210809-075212-marostegui.json [production]
07:50 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
07:45 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
07:30 <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:710919|Enable shellbox for constraint for all of wikidata (T176312)]] (duration: 00m 58s) [production]
07:15 <marostegui> Stop db1117:3323 to clone db1107 - T288197 [production]
07:05 <kart__> Updated cxserver to 2021-08-06-062053-production (T288272) [production]
07:04 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1107.eqiad.wmnet with reason: REIMAGE [production]
07:02 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1107.eqiad.wmnet with reason: REIMAGE [production]
06:53 <kartik@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
06:45 <kartik@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
05:56 <XioNoX> enable cloudsw1-c8 interfaces toward cloudsw2-c8 - T277340 [production]
05:23 <marostegui> Lag in s4 (commonswiki) will appear on clouddb* hosts (wiki replicas) T288273 [production]
05:22 <marostegui> Optimize commonswiki.image on eqiad, lag will appear - T288273 [production]
2021-08-08 §
22:08 <wm-bot> <lucaswerkmeister> END - optimize querytime table [tools.quickcategories]
22:08 <wm-bot> <lucaswerkmeister> START - optimize querytime table [tools.quickcategories]
22:02 <wm-bot> <lucaswerkmeister> END - purge querytime rows older than 30 days, in batches of 1000 sleeping for 1s between batches (deleted 10257486 rows) [tools.quickcategories]
13:16 <wm-bot> <lucaswerkmeister> START - purge querytime rows older than 30 days, in batches of 1000 sleeping for 1s between batches [tools.quickcategories]
07:39 <majavah> stop running and queued GlobalRollbacker-unreviewer grid jobs per private tool maintainer request [tools.dannys712-bot]
2021-08-07 §
20:13 <wm-bot> <lucaswerkmeister> deployed 7028c292e7 (lock less tables) [tools.quickcategories]
19:06 <wm-bot> <lucaswerkmeister> deployed bbdebedd08 (styling improvement) [tools.quickcategories]
05:59 <majavah> restart nginx on toolserver-proxy-01 if that helps with flapping icinga certificate expiry check [tools]
05:30 <Operator873|CVN> restarted CVNBot5 - was not reporting [cvn]
00:48 <James_F> Docker: Publish quibble-buster-php73-coverage 1.1.1 for T287918. [releng]
00:29 <James_F> Zuul: Add skin-coverage jobs to all Wikimedia production skins T287918 [releng]
00:27 <James_F> Zuul: Provide a skin-coverage template T287918 [releng]
2021-08-06 §
23:53 <James_F> Docker: Publishing quibble-buster-php73-coverage 1.1.0 for T287918 [releng]
23:47 <James_F> Zuul: [mediawiki/skins/Mirage] Not a production skin; move to right section [releng]
19:17 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:12 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
19:04 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:53 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
18:53 <cmjohnson@cumin1001> END (ERROR) - Cookbook sre.dns.netbox (exit_code=97) [production]
18:52 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
18:45 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:41 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]