3601-3650 of 10000 results (63ms)
2024-07-24 §
11:06 <dreamyjazz@deploy1002> dreamyjazz: Backport for [[gerrit:1056462|Remove now unused $wgGlobalBlockingDatabase definition (T370856)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
11:03 <dreamyjazz@deploy1002> Started scap sync-world: Backport for [[gerrit:1056462|Remove now unused $wgGlobalBlockingDatabase definition (T370856)]] [production]
11:00 <jiji@deploy1002> Finished scap: Noop, bumping mediawiki chart version (duration: 02m 32s) [production]
10:57 <jiji@deploy1002> Started scap sync-world: Noop, bumping mediawiki chart version [production]
10:54 <jiji@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
10:54 <jiji@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
10:54 <jiji@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
10:53 <jiji@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
10:45 <aborrero@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 [admin]
10:45 <aborrero@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 [admin]
10:44 <aborrero@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 [admin]
10:44 <aborrero@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 [admin]
10:43 <aborrero@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 [admin]
10:43 <aborrero@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 [admin]
10:42 <aborrero@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 [admin]
10:42 <aborrero@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 [admin]
10:33 <akosiaris@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on deploy1003.eqiad.wmnet with reason: host reimage [production]
10:28 <akosiaris@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on deploy1003.eqiad.wmnet with reason: host reimage [production]
10:16 <akosiaris@cumin1002> START - Cookbook sre.hosts.reimage for host deploy1003.eqiad.wmnet with OS bullseye [production]
10:15 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 21 days, 0:00:00 on 16 hosts with reason: Legacy appserver spindown [production]
10:15 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 21 days, 0:00:00 on 16 hosts with reason: Legacy appserver spindown [production]
09:21 <sstefanova@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx [tools]
09:21 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx [tools]
08:58 <sstefanova@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx [toolsbeta]
08:58 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx [toolsbeta]
08:11 <sstefanova@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission [tools]
08:11 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission [tools]
07:07 <sstefanova@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component ingress-admission [tools]
06:57 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission [tools]
06:54 <XioNoX> deploy CR1056198 Rename LVS-service-IPs prefix-list [production]
06:45 <sstefanova@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission [toolsbeta]
06:45 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission [toolsbeta]
06:01 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2173', diff saved to https://phabricator.wikimedia.org/P66908 and previous config saved to /var/cache/conftool/dbconfig/20240724-060142-marostegui.json [production]
05:46 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2173', diff saved to https://phabricator.wikimedia.org/P66907 and previous config saved to /var/cache/conftool/dbconfig/20240724-054635-marostegui.json [production]
05:45 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1150.eqiad.wmnet with reason: Maintenance [production]
05:45 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1150.eqiad.wmnet with reason: Maintenance [production]
05:31 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2173 (T367856)', diff saved to https://phabricator.wikimedia.org/P66906 and previous config saved to /var/cache/conftool/dbconfig/20240724-053128-marostegui.json [production]
05:12 <akosiaris@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host deploy1003.eqiad.wmnet with OS bullseye [production]
01:14 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host pc2017.codfw.wmnet with OS bookworm [production]
00:12 <jclark@cumin1002> START - Cookbook sre.hosts.reimage for host pc1017.eqiad.wmnet with OS bookworm [production]
2024-07-23 §
23:58 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host pc1017.eqiad.wmnet with OS bookworm [production]
23:54 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host pc2017.codfw.wmnet with OS bookworm [production]
23:49 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['pc2017'] [production]
23:43 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['pc2017'] [production]
23:43 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['pc2017'] [production]
23:42 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['pc2017'] [production]
23:41 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host pc2017.mgmt.codfw.wmnet with reboot policy FORCED [production]
23:23 <eileen> civicrm upgraded from 4247715d to c656ab2f [production]
23:20 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host pc2017.mgmt.codfw.wmnet with reboot policy FORCED [production]
23:17 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]