2851-2900 of 10000 results (35ms)
2021-08-26 §
09:24 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-serve-ctrl1001.eqiad.wmnet [production]
09:21 <elukey> elukey@kafka-main1001:~$ kafka acls --add --allow-principal User:CN=varnishkafka --producer --topic statsv - T286038 [production]
09:21 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-serve-ctrl1001.eqiad.wmnet [production]
09:20 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-etcd1003.eqiad.wmnet [production]
09:17 <elukey> restart varnishkafka-statsv on cp4032 to pick up TLS settings [production]
09:15 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-etcd1003.eqiad.wmnet [production]
09:15 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-etcd1002.eqiad.wmnet [production]
09:13 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-etcd1002.eqiad.wmnet [production]
09:12 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-etcd1001.eqiad.wmnet [production]
09:10 <klausman@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-etcd1001.eqiad.wmnet [production]
08:52 <vgutierrez> restart varnishkafka-statsv on cp4032 [production]
06:59 <marostegui@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on db1138.eqiad.wmnet with reason: REIMAGE [production]
06:57 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1138.eqiad.wmnet with reason: REIMAGE [production]
06:48 <godog> more weight to ms-be20[62-65] - T288458 [production]
06:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1160 T288273', diff saved to https://phabricator.wikimedia.org/P17085 and previous config saved to /var/cache/conftool/dbconfig/20210826-064655-marostegui.json [production]
06:43 <marostegui> Reimage s4 eqiad master (db1138), expect lag on eqiad T288803 [production]
06:37 <elukey@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
06:33 <elukey@cumin1001> START - Cookbook sre.dns.netbox [production]
2021-08-25 §
23:23 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
23:22 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
23:20 <urbanecm> Evening B&C window completed [production]
23:19 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/GlobalWatchlist/modules/EntryLog.js: 230aec3fe7f3d0e325882a5fc926e9f3e4e86717: GlobalWatchlistEntryLog: fix storing log id (T288385) (duration: 01m 07s) [production]
22:19 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:11 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:10 <legoktm@deploy1002> Synchronized debug.json: List primary DC servers first (T289246) (duration: 01m 04s) [production]
22:09 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:07 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/Flow/includes/Content/BoardContent.php: 694b94657d251df64145e8153b269094bba75be9: BoardContent: Fix deprecation warning (T289625) (duration: 01m 04s) [production]
22:04 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/VisualEditor/includes/ApiVisualEditor.php: 73478bc9c72286123cef69e57e0aef9e745dcff9: Make sure params is an array (T289730) (duration: 01m 04s) [production]
22:00 <legoktm@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'shellbox-constraints' for release 'main' . [production]
21:59 <brennen> 1.37.0-wmf.20 train status (T281161) blockers should be patched shortly; as we've reached the 15:00 Pacific deploy cutoff for the day, train will resume first thing in US morning [production]
21:58 <legoktm@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'shellbox-constraints' for release 'main' . [production]
21:37 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:36 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:35 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/DiscussionTools/includes/Notifications/EventDispatcher.php: cc04b33dec6b9aed1d7621957c4de527266600d1: EventDispatcher: Try really, really hard to read from master (T289717) (duration: 01m 04s) [production]
21:32 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/includes/page/PageStore.php: 34fb2b99104d0a2bda8aa202f4cdeb07cb983531: PageStore: Pass query flags to getPageByName() (T289717; T195069) (duration: 01m 06s) [production]
21:19 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:17 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:14 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/ConfirmEdit/SimpleCaptcha/SimpleCaptcha.php: 190d8b7579af981cf2f5e4a6d9457ee0a7edca3f: Use Parser::getUserIdentity() instead of ::getUser() in SimpleCaptcha (T289731) (duration: 01m 05s) [production]
21:05 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:04 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:03 <urbanecm@deploy1002> Synchronized php-1.37.0-wmf.20/extensions/ProofreadPage/: 913043a5ca7982e07ab0c01f88076af866a43cc3: Fixes exception thrown by FilePagination::getPageNumber (T289728) (duration: 01m 06s) [production]
20:02 <brennen> 1.37.0-wmf.20 (T281161) status: blocked at group0; 2/3 blockers have probable patches, all seem to be getting attention, so holding off on blocker mail for now. [production]
19:54 <urbanecm> enwikisource: Start server-side upload for one video file (T289698) [production]
19:45 <urbanecm> Start server-side upload for ~2 GB tiff file (T289711) [production]
19:31 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:29 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
19:28 <dancy@deploy1002> Synchronized php: group1 wikis to 1.37.0-wmf.19 (duration: 01m 05s) [production]
19:27 <dancy@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.37.0-wmf.19 [production]
19:17 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]