2451-2500 of 10000 results (33ms)
2024-12-11 §
09:32 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ml-lab1002.eqiad.wmnet with OS bookworm [production]
09:32 <elukey@cumin1002> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - elukey@cumin1002" [production]
09:32 <jelto@cumin1002> START - Cookbook sre.hosts.rename from kubernetes2014 to wikikube-worker2183 [production]
09:31 <jelto@cumin1002> END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from kubernetes2013 to wikikube-worker2182 [production]
09:31 <jelto@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2182 [production]
09:31 <jelto@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2182 [production]
09:30 <jelto@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:30 <jelto@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2013 to wikikube-worker2182 - jelto@cumin1002" [production]
09:30 <jelto@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2013 to wikikube-worker2182 - jelto@cumin1002" [production]
09:26 <jelto@cumin1002> START - Cookbook sre.dns.netbox [production]
09:26 <jelto@cumin1002> START - Cookbook sre.hosts.rename from kubernetes2013 to wikikube-worker2182 [production]
09:26 <jelto@cumin1002> END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from kubernetes2012 to wikikube-worker2181 [production]
09:25 <marostegui@cumin1002> dbctl commit (dc=all): 'db2136 (re)pooling @ 25%: Repooling after upgrade', diff saved to https://phabricator.wikimedia.org/P71696 and previous config saved to /var/cache/conftool/dbconfig/20241211-092535-root.json [production]
09:25 <jelto@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2181 [production]
09:25 <jelto@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2181 [production]
09:25 <jelto@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:25 <jelto@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2012 to wikikube-worker2181 - jelto@cumin1002" [production]
09:24 <jelto@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2012 to wikikube-worker2181 - jelto@cumin1002" [production]
09:20 <jelto@cumin1002> START - Cookbook sre.dns.netbox [production]
09:20 <jelto@cumin1002> START - Cookbook sre.hosts.rename from kubernetes2012 to wikikube-worker2181 [production]
09:20 <jelto@cumin1002> END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from kubernetes2011 to wikikube-worker2180 [production]
09:19 <jelto@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2180 [production]
09:19 <jelto@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2180 [production]
09:19 <jelto@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:19 <jelto@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2011 to wikikube-worker2180 - jelto@cumin1002" [production]
09:18 <jelto@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2011 to wikikube-worker2180 - jelto@cumin1002" [production]
09:14 <jelto@cumin1002> START - Cookbook sre.dns.netbox [production]
09:14 <jelto@cumin1002> START - Cookbook sre.hosts.rename from kubernetes2011 to wikikube-worker2180 [production]
09:12 <wmbot~lucaswerkmeister@tools-bastion-13> deployed 8a1cfeaa7b (Codex 1.18.0) [tools.codex-playground]
09:10 <marostegui@cumin1002> dbctl commit (dc=all): 'db2136 (re)pooling @ 10%: Repooling after upgrade', diff saved to https://phabricator.wikimedia.org/P71695 and previous config saved to /var/cache/conftool/dbconfig/20241211-091029-root.json [production]
09:08 <jelto@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubernetes[2011-2014].codfw.wmnet [production]
09:08 <jelto@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host kubernetes[2011-2014].codfw.wmnet [production]
09:06 <jelto@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubernetes[2011-2014].codfw.wmnet [production]
09:05 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2136 to upgrade MariaDB 10.11 T378940', diff saved to https://phabricator.wikimedia.org/P71694 and previous config saved to /var/cache/conftool/dbconfig/20241211-090538-marostegui.json [production]
09:04 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db2136.codfw.wmnet with reason: maintenance [production]
09:04 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on db2136.codfw.wmnet with reason: maintenance [production]
09:04 <jelto@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host kubernetes[2011-2014].codfw.wmnet [production]
08:12 <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) (T381911) [codesearch]
08:12 <fnegri@cloudcumin1001> START - Cookbook wmcs.openstack.quota_increase (T381911) [codesearch]
02:30 <eileen> civicrm upgraded from 3ef855ca to ddda6d67 [production]
01:36 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T376150, xfer wdqs scholarly 2023(public)->2026(internal)) xfer scholarly_articles from wdqs2023.codfw.wmnet -> wdqs2027.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
00:50 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T376150, xfer wdqs scholarly 2023(public)->2026(internal)) xfer scholarly_articles from wdqs2023.codfw.wmnet -> wdqs2027.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
2024-12-10 §
23:35 <eileen> config revision changed from b3741848 to ca701cba add phone update job [production]
22:54 <jhathaway@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on ms-be1088.eqiad.wmnet with reason: T381919 [production]
22:54 <jhathaway@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on ms-be1088.eqiad.wmnet with reason: T381919 [production]
22:49 <jhathaway@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1088.eqiad.wmnet with OS bookworm [production]
22:36 <cjming> end of UTC late backport window [production]
22:22 <jhathaway@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1088.eqiad.wmnet with reason: host reimage [production]
22:19 <jhathaway@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1088.eqiad.wmnet with reason: host reimage [production]
22:15 <cjming@deploy2002> Finished scap sync-world: Backport for [[gerrit:1100864|Disable stats collection when WMF_MAINTENANCE_OFFLINE is set (T380609)]] (duration: 11m 24s) [production]