1601-1650 of 10000 results (51ms)
2022-03-11 §
06:13 <marostegui> Reboot dbproxy1014 T303174 [production]
06:09 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 50%: After schema change', diff saved to https://phabricator.wikimedia.org/P22362 and previous config saved to /var/cache/conftool/dbconfig/20220311-060913-root.json [production]
05:54 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 25%: After schema change', diff saved to https://phabricator.wikimedia.org/P22361 and previous config saved to /var/cache/conftool/dbconfig/20220311-055409-root.json [production]
05:45 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1106', diff saved to https://phabricator.wikimedia.org/P22360 and previous config saved to /var/cache/conftool/dbconfig/20220311-054514-marostegui.json [production]
02:54 <eileen> revision changed from 9fb68b24 to 252269c8 [production]
01:56 <eileen> civicrm revision changed from 8501c38c to 9fb68b24 [production]
01:31 <eileen> civicrm changed from 4cb2bdbc to 8501c38c [production]
00:33 <TimStarling> on mwmaint1002 running populateGlobalEditCount.php [production]
00:03 <rzl@deploy1002> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
00:01 <rzl@deploy1002> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
2022-03-10 §
23:58 <rzl@deploy1002> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
23:55 <rzl@deploy1002> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
23:08 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
23:07 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
22:42 <tstarling@deploy1002> Finished scap: global_edit_count gerrit 769561 (duration: 15m 12s) [production]
22:27 <tstarling@deploy1002> Started scap: global_edit_count gerrit 769561 [production]
22:24 <tstarling@deploy1002> Synchronized php-1.38.0-wmf.25/extensions/CentralAuth/includes/User/CentralAuthUser.php: global_edit_count gerrit 769561 (duration: 00m 47s) [production]
22:24 <tstarling@deploy1002> Synchronized php-1.38.0-wmf.25/extensions/CentralAuth/includes/Hooks/Handlers/UserEditCountUpdateHookHandler.php: global_edit_count gerrit 769561 (duration: 00m 47s) [production]
22:23 <tstarling@deploy1002> Synchronized php-1.38.0-wmf.25/extensions/CentralAuth/includes/CentralAuthServices.php: global_edit_count gerrit 769561 (duration: 00m 47s) [production]
22:22 <tstarling@deploy1002> Synchronized php-1.38.0-wmf.25/extensions/CentralAuth/includes/ServiceWiring.php: global_edit_count gerrit 769561 (duration: 00m 48s) [production]
22:21 <tstarling@deploy1002> Synchronized php-1.38.0-wmf.25/extensions/CentralAuth/includes/CentralAuthEditCounter.php: global_edit_count gerrit 769561 (duration: 00m 48s) [production]
22:10 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
22:10 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
22:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
22:08 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
22:08 <bking@cumin1001> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:05 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:04 <bking@cumin1001> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:04 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:02 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
22:02 <bking@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: relforge cluster restart - bking@cumin1001 - T301955 [production]
21:43 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:42 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:42 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:41 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:41 <rzl> UTC late B&C training window done [production]
21:39 <rzl@deploy1002> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:769779|CommonSettings: Update comment about Image Suggestions API (T294362)]] (duration: 00m 48s) [production]
21:34 <rzl@deploy1002> Synchronized php-1.38.0-wmf.25/extensions/DiscussionTools/modules/controller.js: Backport: [[gerrit:769559|Fix highlighting of comments when reloading (T303261)]] (duration: 00m 47s) [production]
21:33 <rzl@deploy1002> Synchronized php-1.38.0-wmf.25/extensions/VisualEditor/modules/ve-mw: Backport: [[gerrit:769558|Preserve classes on media wrapper links (T292657 T303469)]] (duration: 00m 49s) [production]
21:25 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:25 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:25 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:24 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:19 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:18 <cstone> update Donation Interface revision changed from ca37a93e to 5db12b21 [production]
21:18 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:17 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:13 <rzl@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:766307|Remove centralauth-oversight from the config (T302675)]] (duration: 00m 49s) [production]
21:12 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]