1601-1650 of 10000 results (69ms)
2022-08-23 §
02:08 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
02:08 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
01:41 <TimStarling> on mw1411, mw1413, mw1419, mw1429, mw1431, mw1433: set energy_performance_preference to balance_performance T315398 [production]
01:11 <TimStarling> on mw1411, mw1413, mw1419, mw1429, mw1431, mw1433: set scaling_governor to powersave and energy_performance_preference to performance [production]
00:09 <pt1979@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1187.eqiad.wmnet with OS bullseye [production]
2022-08-22 §
23:55 <pt1979@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1187.eqiad.wmnet with reason: host reimage [production]
23:52 <pt1979@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1187.eqiad.wmnet with reason: host reimage [production]
23:39 <pt1979@cumin1001> START - Cookbook sre.hosts.reimage for host db1187.eqiad.wmnet with OS bullseye [production]
23:10 <tstarling@puppetmaster1001> conftool action : set/pooled=true; selector: name=codfw,dnsdisc=(appservers|api)-ro [production]
23:04 <TimStarling> Re-enable multi-DC mode on testwiki, test2wiki and mediawiki.org [production]
21:56 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge elasticsearch and plugin upgrade - bking@cumin2002 - T315604 [production]
21:55 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge elasticsearch and plugin upgrade - bking@cumin2002 - T315604 [production]
21:46 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge elasticsearch and plugin upgrade - bking@cumin2002 - T315604 [production]
21:45 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge elasticsearch and plugin upgrade - bking@cumin2002 - T315604 [production]
21:26 <sbassett> Deployed security fix for T310763 [production]
21:17 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge elasticsearch and plugin upgrade - bking@cumin2002 - T315604 [production]
21:17 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: relforge elasticsearch and plugin upgrade - bking@cumin2002 - T315604 [production]
21:11 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc-wf2002.codfw.wmnet with OS bullseye [production]
21:06 <pt1979@cumin1001> END (FAIL) - Cookbook sre.hosts.dhcp (exit_code=99) for host db1185.eqiad.wmnet [production]
21:04 <pt1979@cumin1001> START - Cookbook sre.hosts.dhcp for host db1185.eqiad.wmnet [production]
21:02 <pt1979@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db1185.eqiad.wmnet with OS bullseye [production]
21:01 <pt1979@cumin1001> START - Cookbook sre.hosts.reimage for host db1185.eqiad.wmnet with OS bullseye [production]
20:59 <pt1979@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db1185.eqiad.wmnet with OS bullseye [production]
20:59 <pt1979@cumin1001> START - Cookbook sre.hosts.reimage for host db1185.eqiad.wmnet with OS bullseye [production]
20:59 <pt1979@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db1195.eqiad.wmnet with OS bullseye [production]
20:58 <pt1979@cumin1001> START - Cookbook sre.hosts.reimage for host db1195.eqiad.wmnet with OS bullseye [production]
20:56 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc-wf2002.codfw.wmnet with reason: host reimage [production]
20:53 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mc-wf2002.codfw.wmnet with reason: host reimage [production]
20:51 <urbanecm@deploy1002> Synchronized php-1.39.0-wmf.25/skins/Vector/: e0ff7634ac529acec6d298992b45b23203b682c1: Layout: Restore disabling of max width on certain pages (T315460) (duration: 03m 37s) [production]
20:39 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:38 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:38 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:37 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:33 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host mc-wf2002.codfw.wmnet with OS bullseye [production]
20:28 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc-wf2001.codfw.wmnet with OS bullseye [production]
20:13 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc-wf2001.codfw.wmnet with reason: host reimage [production]
20:09 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mc-wf2001.codfw.wmnet with reason: host reimage [production]
20:04 <xcollazo@deploy1002> Finished deploy [airflow-dags/analytics@5ac442f]: Use instance specific HDFS cache on analytics (duration: 00m 40s) [production]
20:03 <xcollazo@deploy1002> Started deploy [airflow-dags/analytics@5ac442f]: Use instance specific HDFS cache on analytics [production]
19:50 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host mc-wf2001.codfw.wmnet with OS bullseye [production]
19:35 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['mc-wf2002'] [production]
19:28 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['mc-wf2002'] [production]
19:27 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['mc-wf2001'] [production]
19:20 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['mc-wf2001'] [production]
19:12 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mc-wf2002.mgmt.codfw.wmnet with reboot policy FORCED [production]
19:11 <xcollazo@deploy1002> Finished deploy [airflow-dags/analytics_test@5ac442f]: Use instance specific HDFS cache on analytics_test (duration: 00m 17s) [production]
19:11 <xcollazo@deploy1002> Started deploy [airflow-dags/analytics_test@5ac442f]: Use instance specific HDFS cache on analytics_test [production]
19:04 <xcollazo@deploy1002> Finished deploy [airflow-dags/analytics_test@9edd1ab]: Use instance specific HDFS cache on analytics_test (duration: 00m 05s) [production]
19:04 <xcollazo@deploy1002> Started deploy [airflow-dags/analytics_test@9edd1ab]: Use instance specific HDFS cache on analytics_test [production]
18:59 <xcollazo@deploy1002> Finished deploy [airflow-dags/platform_eng@5ac442f]: Use instance specific HDFS cache on platform_eng (duration: 00m 10s) [production]