451-500 of 10000 results (86ms)
2024-01-10 §
11:37 <dani@deploy2002> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
11:37 <dani@deploy2002> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
11:36 <dani@deploy2002> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
11:36 <dani@deploy2002> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
11:03 <moritzm> installing PHP 7.3 security updates [production]
10:46 <moritzm> installing curl security updates [production]
10:04 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts testreduce1001.eqiad.wmnet [production]
10:04 <jmm@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
10:04 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: testreduce1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" [production]
10:02 <jmm@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: testreduce1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" [production]
10:01 <jgiannelos@deploy2002> helmfile [codfw] DONE helmfile.d/services/wikifeeds: sync [production]
10:00 <jgiannelos@deploy2002> helmfile [codfw] START helmfile.d/services/wikifeeds: sync [production]
10:00 <jgiannelos@deploy2002> helmfile [eqiad] DONE helmfile.d/services/wikifeeds: sync [production]
10:00 <jgiannelos@deploy2002> helmfile [eqiad] START helmfile.d/services/wikifeeds: sync [production]
09:57 <jmm@cumin2002> START - Cookbook sre.dns.netbox [production]
09:55 <hashar@deploy2002> Finished deploy [integration/docroot@355ddbb]: (no justification provided) (duration: 00m 04s) [production]
09:55 <hashar@deploy2002> Started deploy [integration/docroot@355ddbb]: (no justification provided) [production]
09:55 <moritzm> installing git security updates on deployment hosts [production]
09:53 <hashar@deploy2002> Finished deploy [integration/docroot@355ddbb]: Dummy deploy to test git safe.directory # T335354 (duration: 00m 06s) [production]
09:53 <jmm@cumin2002> START - Cookbook sre.hosts.decommission for hosts testreduce1001.eqiad.wmnet [production]
09:53 <hashar@deploy2002> Started deploy [integration/docroot@355ddbb]: Dummy deploy to test git safe.directory # T335354 [production]
09:38 <akosiaris@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mw1349.eqiad.wmnet with reason: Trying to reproduce wdat_wdt watchdog problem [production]
09:38 <akosiaris@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on mw1349.eqiad.wmnet with reason: Trying to reproduce wdat_wdt watchdog problem [production]
09:38 <akosiaris@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mw1378.eqiad.wmnet with reason: Trying to reproduce wdat_wdt watchdog problem [production]
09:38 <akosiaris@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on mw1378.eqiad.wmnet with reason: Trying to reproduce wdat_wdt watchdog problem [production]
09:01 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 15133 [production]
09:00 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'configure' for AS: 15133 [production]
08:59 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 13150 [production]
08:57 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'configure' for AS: 13150 [production]
08:47 <dcausse@deploy2002> Finished scap: Backport for [[gerrit:989442|enable page_rerender for 4th batch of wikis (T351503)]] (duration: 11m 50s) [production]
08:42 <akosiaris@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mw1349.eqiad.wmnet with reason: Trying to reproduce wdat_wdt watchdog problem [production]
08:41 <akosiaris@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on mw1349.eqiad.wmnet with reason: Trying to reproduce wdat_wdt watchdog problem [production]
08:41 <moritzm> installing Exim security updates [production]
08:40 <dcausse@deploy2002> pfischer and dcausse: Continuing with sync [production]
08:37 <dcausse@deploy2002> pfischer and dcausse: Backport for [[gerrit:989442|enable page_rerender for 4th batch of wikis (T351503)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
08:35 <dcausse@deploy2002> Started scap: Backport for [[gerrit:989442|enable page_rerender for 4th batch of wikis (T351503)]] [production]
08:12 <kartik@deploy2002> Finished scap: Backport for [[gerrit:988984|testwiki: Enable Section translation on WPs with Content Translation available as default (T351882)]] (duration: 09m 10s) [production]
08:06 <kartik@deploy2002> kartik: Continuing with sync [production]
08:04 <kartik@deploy2002> kartik: Backport for [[gerrit:988984|testwiki: Enable Section translation on WPs with Content Translation available as default (T351882)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
08:03 <kartik@deploy2002> Started scap: Backport for [[gerrit:988984|testwiki: Enable Section translation on WPs with Content Translation available as default (T351882)]] [production]
07:53 <moritzm> installing openjdk-8 security updates [production]
07:15 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2143.codfw.wmnet with OS bookworm [production]
06:53 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2143.codfw.wmnet with reason: host reimage [production]
06:50 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on db2143.codfw.wmnet with reason: host reimage [production]
06:32 <marostegui@cumin1002> START - Cookbook sre.hosts.reimage for host db2143.codfw.wmnet with OS bookworm [production]
2024-01-09 §
21:33 <ryankemper@cumin2002> END (FAIL) - Cookbook sre.wdqs.restart (exit_code=99) [production]
21:23 <aqu@deploy2002> Finished deploy [airflow-dags/analytics@ea53374]: Regular airflow-dags/analytics weekly train [airflow-dags@ea53374f] (duration: 00m 28s) [production]
21:22 <aqu@deploy2002> Started deploy [airflow-dags/analytics@ea53374]: Regular airflow-dags/analytics weekly train [airflow-dags@ea53374f] [production]
21:21 <aqu@deploy2002> Finished deploy [airflow-dags/analytics_test@ea53374]: Regular airflow-dags/analytics_test weekly train [airflow-dags@ea53374f] (duration: 00m 12s) [production]
21:21 <aqu@deploy2002> Started deploy [airflow-dags/analytics_test@ea53374]: Regular airflow-dags/analytics_test weekly train [airflow-dags@ea53374f] [production]