7201-7250 of 10000 results (43ms)
2021-04-13 ยง
22:41 <mutante> welcome new deployer Silvan Heintze (sihe) (T279764) [production]
22:40 <cstone> civicrm revision changed from 76bd8ff009 to a4c1a7b842 [production]
22:07 <ejegg> updated payments-wiki from 70f5163816 to 9a4eef1375 [production]
22:06 <dzahn@cumin1001> conftool action : set/weight=15; selector: name=mw2395.codfw.wmnet,cluster=jobrunner [production]
22:06 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2395.codfw.wmnet,cluster=jobrunner [production]
22:04 <Urbanecm> [urbanecm@mwmaint1002 /srv/mediawiki]$ foreachwikiindblist growthexperiments sql.php php-1.37.0-wmf.1/extensions/GrowthExperiments/maintenance/schemas/mysql/growthexperiments_mentor_mentee.sql # T278573 [production]
21:50 <dzahn@cumin1001> conftool action : set/weight=15; selector: name=mw2394.codfw.wmnet,cluster=jobrunner [production]
21:49 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2394.codfw.wmnet,cluster=jobrunner [production]
21:48 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw2394.codfw.wmnet,service=jobrunner [production]
21:45 <mutante> mw2394, mw2395 - scap pull [production]
21:42 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw2395.codfw.wmnet [production]
21:35 <mutante> mw2394 - rebooting [production]
21:34 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw2394.codfw.wmnet [production]
21:23 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2395.codfw.wmnet with reason: REIMAGE [production]
21:21 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2395.codfw.wmnet with reason: REIMAGE [production]
21:15 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2394.codfw.wmnet with reason: REIMAGE [production]
21:13 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2394.codfw.wmnet with reason: REIMAGE [production]
20:58 <mutante> mw2395, mw2395 - reimaging as jobrunners (T279100) [production]
20:56 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw[2394-2395].codfw.wmnet with reason: reimage [production]
20:56 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw[2394-2395].codfw.wmnet with reason: reimage [production]
20:47 <mutante> [kubemaster1001:~] $ sudo kubectl delete pod linkrecommendation-production-load-datasets-1618311600-hn6k8 -n linkrecommendation (T280076) [production]
19:35 <kharlan@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . [production]
19:35 <kharlan@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . [production]
19:32 <kharlan@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . [production]
19:32 <kharlan@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . [production]
19:29 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wtp1033.eqiad.wmnet with reason: REIMAGE [production]
19:28 <kharlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'linkrecommendation' for release 'staging' . [production]
19:27 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wtp1032.eqiad.wmnet with reason: REIMAGE [production]
19:27 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on wtp1033.eqiad.wmnet with reason: REIMAGE [production]
19:25 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on wtp1032.eqiad.wmnet with reason: REIMAGE [production]
19:17 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on parse2020.codfw.wmnet with reason: REIMAGE [production]
19:15 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on parse2020.codfw.wmnet with reason: REIMAGE [production]
19:11 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.37.0-wmf.1 [production]
18:45 <jhuneidi@deploy1002> Pruned MediaWiki: 1.36.0-wmf.37 (duration: 03m 16s) [production]
18:11 <jhuneidi@deploy1002> Finished scap: testwikis wikis to 1.37.0-wmf.1 (duration: 30m 36s) [production]
18:01 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wtp1031.eqiad.wmnet with reason: REIMAGE [production]
17:59 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on wtp1031.eqiad.wmnet with reason: REIMAGE [production]
17:58 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wtp1030.eqiad.wmnet with reason: REIMAGE [production]
17:56 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on wtp1030.eqiad.wmnet with reason: REIMAGE [production]
17:54 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on parse2019.codfw.wmnet with reason: REIMAGE [production]
17:54 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.cf (exit_code=0) [production]
17:54 <ayounsi@cumin1001> START - Cookbook sre.network.cf [production]
17:52 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on parse2018.codfw.wmnet with reason: REIMAGE [production]
17:52 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on parse2019.codfw.wmnet with reason: REIMAGE [production]
17:50 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on parse2017.codfw.wmnet with reason: REIMAGE [production]
17:50 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on parse2018.codfw.wmnet with reason: REIMAGE [production]
17:48 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on parse2017.codfw.wmnet with reason: REIMAGE [production]
17:41 <jhuneidi@deploy1002> Started scap: testwikis wikis to 1.37.0-wmf.1 [production]
17:29 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.cf (exit_code=0) [production]
17:28 <ayounsi@cumin1001> START - Cookbook sre.network.cf [production]