2201-2250 of 10000 results (62ms)
2022-06-01 ยง
19:35 <ebysans@deploy1002> Started deploy [analytics/refinery@13f791b] (thin): Regular analytics weekly train THIN [analytics/refinery@13f791b] [production]
19:19 <ebysans@deploy1002> Finished deploy [analytics/refinery@13f791b]: Regular analytics weekly train [analytics/refinery@13f791b] (duration: 23m 12s) [production]
18:56 <ebysans@deploy1002> Started deploy [analytics/refinery@13f791b]: Regular analytics weekly train [analytics/refinery@13f791b] [production]
18:52 <SandraEbele> About to deploy analytics/refinery (weekly deployment train) [production]
18:14 <jhuneidi@deploy1002> Synchronized php: group1 wikis to 1.39.0-wmf.14 refs T308067 (duration: 03m 02s) [production]
18:12 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
18:11 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.39.0-wmf.14 refs T308067 [production]
18:11 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
18:11 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
18:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
18:05 <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.14/extensions/Wikibase/client: Backport: [[gerrit:802114|Don't call saveSettings in EchoNotificationsHandlers::doLocalUserCreated (T306636)]] (duration: 03m 11s) [production]
18:00 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
17:59 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
17:59 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
17:58 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
15:58 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1047.eqiad.wmnet with OS bullseye [production]
15:43 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1047.eqiad.wmnet with reason: host reimage [production]
15:40 <mvernon@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1047.eqiad.wmnet with reason: host reimage [production]
15:24 <mvernon@cumin2002> START - Cookbook sre.hosts.reimage for host ms-be1047.eqiad.wmnet with OS bullseye [production]
15:02 <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.14/extensions/Thanks/includes/Hooks.php: Backport: [[gerrit:802110|Don't call saveOptions in Hooks::onAccountCreated (T306636)]] (duration: 03m 10s) [production]
15:01 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
15:00 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
15:00 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:59 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
14:55 <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.13/extensions/Thanks/includes/Hooks.php: Backport: [[gerrit:802109|Don't call saveOptions in Hooks::onAccountCreated (T306636)]] (duration: 03m 10s) [production]
14:54 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
14:54 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
14:54 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:53 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
14:46 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1046.eqiad.wmnet with OS bullseye [production]
14:31 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1046.eqiad.wmnet with reason: host reimage [production]
14:28 <mvernon@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1046.eqiad.wmnet with reason: host reimage [production]
14:25 <aikochou@deploy1002> Finished deploy [ores/deploy@3d541df]: Deploy revscoring 2.11.4 to ORES - T309536 (duration: 45m 07s) [production]
14:10 <mvernon@cumin2002> START - Cookbook sre.hosts.reimage for host ms-be1046.eqiad.wmnet with OS bullseye [production]
13:40 <aikochou@deploy1002> Started deploy [ores/deploy@3d541df]: Deploy revscoring 2.11.4 to ORES - T309536 [production]
13:32 <kevinbazira@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
13:32 <kevinbazira@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
12:41 <moritzm> installing ruby-nokogiri security updates [production]
12:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136 (T309617)', diff saved to https://phabricator.wikimedia.org/P29320 and previous config saved to /var/cache/conftool/dbconfig/20220601-122426-ladsgroup.json [production]
12:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P29318 and previous config saved to /var/cache/conftool/dbconfig/20220601-120921-ladsgroup.json [production]
11:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P29317 and previous config saved to /var/cache/conftool/dbconfig/20220601-115416-ladsgroup.json [production]
11:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1137 in x1 with minimal weight to test 10.6.8 T309679 ', diff saved to https://phabricator.wikimedia.org/P29315 and previous config saved to /var/cache/conftool/dbconfig/20220601-114418-marostegui.json [production]
11:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136 (T309617)', diff saved to https://phabricator.wikimedia.org/P29314 and previous config saved to /var/cache/conftool/dbconfig/20220601-113911-ladsgroup.json [production]
11:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1136 (T309617)', diff saved to https://phabricator.wikimedia.org/P29313 and previous config saved to /var/cache/conftool/dbconfig/20220601-113017-ladsgroup.json [production]
11:30 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1136.eqiad.wmnet with reason: Maintenance [production]
11:30 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1136.eqiad.wmnet with reason: Maintenance [production]
11:21 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
11:21 <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.13/extensions/PageTriage/includes/Hooks.php: Backport: [[gerrit:802107|Don't call saveOptions in LocalUserCreated (T306636)]] (duration: 03m 16s) [production]
11:20 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
11:20 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]