1-50 of 10000 results (2ms)
2026-10-09 ยง
18:54 <cscott@deploy1003> Finished scap sync-world: Backport for [[gerrit:1354042|Undeploy Parsoid readviews on frwikisource (T440695)]] (duration: 09m 58s) [production]
18:49 <cscott@deploy1003> cscott, ssastry: Continuing with deployment [production]
18:48 <cscott@deploy1003> cscott, ssastry: Backport for [[gerrit:1354042|Undeploy Parsoid readviews on frwikisource (T440695)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
18:44 <cscott@deploy1003> Started scap sync-world: Backport for [[gerrit:1354042|Undeploy Parsoid readviews on frwikisource (T440695)]] [production]
18:10 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
18:10 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:38 <Daimona> Run database query from T440557#12411029 in x1.wikishared [production]
17:37 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:37 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:37 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:37 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:37 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:37 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:35 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:35 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:33 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:33 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:32 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:32 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:32 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:32 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:31 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:31 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.3.1 (T414595) [tools]
17:30 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.1.3 (T414595) [tools]
17:30 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/kube-state-metrics:v8.1.3 (T414595) [tools]
16:47 <cmooney@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on netbox1003.eqiad.wmnet with reason: Restoring DB from backup on netboxdb1003 [production]
16:05 <cmooney@cumin1004> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox [production]
16:01 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.vps.delete_project (exit_code=0) for project baglama2 in eqiad1 (T440700) [baglama2]
15:57 <andrew@cloudcumin1001> START - Cookbook wmcs.vps.delete_project for project baglama2 in eqiad1 (T440700) [baglama2]
15:50 <cmooney@cumin1004> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:50 <cmooney@cumin1004> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: merge new dns entries for eqiad ab links - cmooney@cumin1004" [production]
15:50 <cmooney@cumin1004> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: merge new dns entries for eqiad ab links - cmooney@cumin1004" [production]
15:46 <cmooney@cumin1004> START - Cookbook sre.dns.netbox [production]
15:44 <cmooney@cumin1004> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox [production]
15:44 <cmooney@cumin1004> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary [production]
15:42 <cmooney@cumin1004> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary [production]
15:42 <oblivian@cumin1004> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for wikikube-worker1081.eqiad.wmnet [production]
15:42 <oblivian@cumin1004> START - Cookbook sre.hosts.remove-downtime for wikikube-worker1081.eqiad.wmnet [production]
15:42 <oblivian@cumin1004> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker1081.eqiad.wmnet [production]
15:42 <oblivian@cumin1004> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker1081.eqiad.wmnet [production]
15:42 <oblivian@cumin1004> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for wikikube-worker1057.eqiad.wmnet [production]
15:42 <oblivian@cumin1004> START - Cookbook sre.hosts.remove-downtime for wikikube-worker1057.eqiad.wmnet [production]
15:33 <larssandergreen> civicrm upgraded from 9e1b80cc to 02ccc9c7 [fundraising]
15:15 <godog> delete trove db baglama2 - T440700 [baglama2]
15:10 <wm-bot2> Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/37949440447 (https://github.com/cluebotng/component-configs/commits/902367266d74fd0d5bf2343e1502e8e1eefc330c) [tools.cluebot3]
14:40 <fnegri@deploy1003> helmfile [eqiad] DONE helmfile.d/services/toolhub: apply [production]
14:40 <fnegri@deploy1003> helmfile [eqiad] START helmfile.d/services/toolhub: apply [production]
14:40 <fnegri@deploy1003> helmfile [codfw] DONE helmfile.d/services/toolhub: apply [production]
14:40 <fnegri@deploy1003> helmfile [codfw] START helmfile.d/services/toolhub: apply [production]
14:39 <fnegri@deploy1003> helmfile [staging] DONE helmfile.d/services/toolhub: apply [production]