5051-5100 of 10000 results (44ms)
2021-08-26 §
11:50 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:48 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:42 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:40 <Lucas_WMDE> EU backport+config window done [production]
11:40 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:39 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.37.0-wmf.19/extensions/Math/src/HookHandlers/ParserHooksHandler.php: Backport: [[gerrit:714853|Allow rendering of <math>0</math> (T288846)]] (duration: 01m 04s) [production]
11:35 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/Math/src/HookHandlers/ParserHooksHandler.php: Backport: [[gerrit:714854|Allow rendering of <math>0</math> (T288846)]] (duration: 01m 05s) [production]
11:32 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host durum1001.eqiad.wmnet [production]
11:21 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm for new host durum1001.eqiad.wmnet [production]
11:20 <nikerabbit@deploy1002> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:714770|Rename wgTranslateBlacklist to wgTranslateDisabledTargetLanguages]] (duration: 01m 05s) [production]
11:13 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:12 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
10:09 <vgutierrez> rolling restart of varnishkafka-statsv - T289618 [production]
10:07 <vgutierrez> disable puppet on cp-text to merge I52cf2a573980e33487d1f05f19b192ae7d13d717 - T286038 [production]
10:06 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-serve1001.eqiad.wmnet [production]
10:01 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-serve1001.eqiad.wmnet [production]
09:36 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-serve1002.eqiad.wmnet [production]
09:30 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-serve1002.eqiad.wmnet [production]
09:24 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-serve-ctrl1001.eqiad.wmnet [production]
09:21 <elukey> elukey@kafka-main1001:~$ kafka acls --add --allow-principal User:CN=varnishkafka --producer --topic statsv - T286038 [production]
09:21 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-serve-ctrl1001.eqiad.wmnet [production]
09:20 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-etcd1003.eqiad.wmnet [production]
09:17 <elukey> restart varnishkafka-statsv on cp4032 to pick up TLS settings [production]
09:15 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-etcd1003.eqiad.wmnet [production]
09:15 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-etcd1002.eqiad.wmnet [production]
09:13 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-etcd1002.eqiad.wmnet [production]
09:12 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-etcd1001.eqiad.wmnet [production]
09:10 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-etcd1001.eqiad.wmnet [production]
08:52 <vgutierrez> restart varnishkafka-statsv on cp4032 [production]
06:59 <marostegui@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1138.eqiad.wmnet with reason: REIMAGE [production]
06:57 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1138.eqiad.wmnet with reason: REIMAGE [production]
06:48 <godog> more weight to ms-be20[62-65] - T288458 [production]
06:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1160 T288273', diff saved to https://phabricator.wikimedia.org/P17085 and previous config saved to /var/cache/conftool/dbconfig/20210826-064655-marostegui.json [production]
06:43 <marostegui> Reimage s4 eqiad master (db1138), expect lag on eqiad T288803 [production]
06:37 <elukey@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
06:33 <elukey@cumin1001> START - Cookbook sre.dns.netbox [production]
2021-08-25 §
23:23 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
23:22 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
23:20 <urbanecm> Evening B&C window completed [production]
23:19 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/GlobalWatchlist/modules/EntryLog.js: 230aec3fe7f3d0e325882a5fc926e9f3e4e86717: GlobalWatchlistEntryLog: fix storing log id (T288385) (duration: 01m 07s) [production]
22:19 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:11 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:10 <legoktm@deploy1002> Synchronized debug.json: List primary DC servers first (T289246) (duration: 01m 04s) [production]
22:09 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:07 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/Flow/includes/Content/BoardContent.php: 694b94657d251df64145e8153b269094bba75be9: BoardContent: Fix deprecation warning (T289625) (duration: 01m 04s) [production]
22:04 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/VisualEditor/includes/ApiVisualEditor.php: 73478bc9c72286123cef69e57e0aef9e745dcff9: Make sure params is an array (T289730) (duration: 01m 04s) [production]
22:00 <legoktm@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'shellbox-constraints' for release 'main' . [production]
21:59 <brennen> 1.37.0-wmf.20 train status (T281161) blockers should be patched shortly; as we've reached the 15:00 Pacific deploy cutoff for the day, train will resume first thing in US morning [production]
21:58 <legoktm@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'shellbox-constraints' for release 'main' . [production]