|
2024-12-20
§
|
| 11:07 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1010.eqiad.wmnet with reason: host reimage |
[production] |
| 11:05 |
<btullis> |
restarted hadoop-mapreduce-historyserver.service on an-master1003 for T382575 |
[analytics] |
| 11:04 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1010.eqiad.wmnet with reason: host reimage |
[production] |
| 10:51 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.reimage for host wikikube-worker1003.eqiad.wmnet with OS bookworm |
[production] |
| 10:49 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1002.eqiad.wmnet with OS bookworm |
[production] |
| 10:46 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.reimage for host wikikube-worker1010.eqiad.wmnet with OS bookworm |
[production] |
| 10:45 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1009.eqiad.wmnet with OS bookworm |
[production] |
| 10:30 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1002.eqiad.wmnet with reason: host reimage |
[production] |
| 10:27 |
<fnegri@cumin1002> |
END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) for database idwikivoyage (T381079) |
[production] |
| 10:27 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1002.eqiad.wmnet with reason: host reimage |
[production] |
| 10:27 |
<fnegri@cumin1002> |
START - Cookbook sre.wikireplicas.add-wiki for database idwikivoyage (T381079) |
[production] |
| 10:25 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1009.eqiad.wmnet with reason: host reimage |
[production] |
| 10:22 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1009.eqiad.wmnet with reason: host reimage |
[production] |
| 10:11 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.reimage for host wikikube-worker1002.eqiad.wmnet with OS bookworm |
[production] |
| 10:09 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1001.eqiad.wmnet with OS bookworm |
[production] |
| 10:05 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.reimage for host wikikube-worker1009.eqiad.wmnet with OS bookworm |
[production] |
| 10:04 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1008.eqiad.wmnet with OS bookworm |
[production] |
| 09:48 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1001.eqiad.wmnet with reason: host reimage |
[production] |
| 09:45 |
<jayme@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1008.eqiad.wmnet with reason: host reimage |
[production] |
| 09:43 |
<moritzm> |
imported imposm3 0.11.1-1+deb12u1 to apt.wikimedia.org T381565 |
[production] |
| 09:43 |
<moritzm> |
imported imposm3 0.11.1-1+deb12u1 to apt.wikimedia.org |
[production] |
| 09:42 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1001.eqiad.wmnet with reason: host reimage |
[production] |
| 09:41 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1008.eqiad.wmnet with reason: host reimage |
[production] |
| 09:23 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.reimage for host wikikube-worker1008.eqiad.wmnet with OS bookworm |
[production] |
| 09:23 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.roll-reimage-nodes rolling reimage on P{wikikube-worker[1008-1011].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) |
[production] |
| 09:23 |
<jayme@cumin1002> |
START - Cookbook sre.hosts.reimage for host wikikube-worker1001.eqiad.wmnet with OS bookworm |
[production] |
| 09:22 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.roll-reimage-nodes rolling reimage on P{wikikube-worker[1001-1004].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) |
[production] |
| 09:21 |
<jayme@cumin1002> |
END (FAIL) - Cookbook sre.k8s.roll-reimage-nodes (exit_code=1) rolling reimage on P{wikikube-worker[1001-1004].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) |
[production] |
| 09:20 |
<jayme@cumin1002> |
END (FAIL) - Cookbook sre.k8s.roll-reimage-nodes (exit_code=1) rolling reimage on P{wikikube-worker[1008-1011].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) |
[production] |
| 09:06 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.roll-reimage-nodes rolling reimage on P{wikikube-worker[1008-1011].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) |
[production] |
| 09:06 |
<jayme@cumin1002> |
START - Cookbook sre.k8s.roll-reimage-nodes rolling reimage on P{wikikube-worker[1001-1004].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) |
[production] |
| 05:37 |
<tchin@deploy2002> |
Finished deploy [airflow-dags/analytics@bcf6276]: (no justification provided) (duration: 04m 11s) |
[production] |
|
2024-12-19
§
|
| 23:51 |
<eileen> |
config revision changed from 404bbbd5 to c447228e |
[production] |
| 21:57 |
<tgr|away> |
UTC late deploys done |
[production] |
| 21:53 |
<tgr@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1105735|Make AuthManagerAutoConfig configuration key more distinctive (T369180)]], [[gerrit:1105739|SUL3: Disable more auth providers in the local leg of SUL3 login (T369180)]], [[gerrit:1105742|[noop] Update private/readme.php (T369180)]], [[gerrit:1105367|Enable $wgWMEStatsBeaconUri (T355837)]] (duration: 21m 34s) |
[production] |
| 21:48 |
<tgr@deploy2002> |
krinkle, tgr: Continuing with sync |
[production] |
| 21:37 |
<tgr@deploy2002> |
krinkle, tgr: Backport for [[gerrit:1105735|Make AuthManagerAutoConfig configuration key more distinctive (T369180)]], [[gerrit:1105739|SUL3: Disable more auth providers in the local leg of SUL3 login (T369180)]], [[gerrit:1105742|[noop] Update private/readme.php (T369180)]], [[gerrit:1105367|Enable $wgWMEStatsBeaconUri (T355837)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:31 |
<tgr@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1105735|Make AuthManagerAutoConfig configuration key more distinctive (T369180)]], [[gerrit:1105739|SUL3: Disable more auth providers in the local leg of SUL3 login (T369180)]], [[gerrit:1105742|[noop] Update private/readme.php (T369180)]], [[gerrit:1105367|Enable $wgWMEStatsBeaconUri (T355837)]] |
[production] |
| 21:29 |
<tgr|away> |
deploying PrivateSettings change 95517e85 |
[production] |
| 21:14 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1105788|Set Flow to read-only on phase 1 wikis (T378833)]], [[gerrit:1105027|Reader Survey: Undeploy (T378660)]] (duration: 11m 14s) |
[production] |
| 21:09 |
<urbanecm@deploy2002> |
urbanecm, kemayo, dani: Continuing with sync |
[production] |
| 21:07 |
<urbanecm@deploy2002> |
urbanecm, kemayo, dani: Backport for [[gerrit:1105788|Set Flow to read-only on phase 1 wikis (T378833)]], [[gerrit:1105027|Reader Survey: Undeploy (T378660)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:03 |
<urbanecm@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1105788|Set Flow to read-only on phase 1 wikis (T378833)]], [[gerrit:1105027|Reader Survey: Undeploy (T378660)]] |
[production] |
| 20:38 |
<jclark@cumin1002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcontrol1011.eqiad.wmnet with OS bookworm |
[production] |
| 19:51 |
<jforrester@deploy2002> |
Finished deploy [integration/docroot@4701376]: I1ea9f34dc6176da4cca5da50c293bd5ff62661b8 for T233089 (duration: 00m 10s) |
[production] |
| 19:51 |
<jforrester@deploy2002> |
Started deploy [integration/docroot@4701376]: I1ea9f34dc6176da4cca5da50c293bd5ff62661b8 for T233089 |
[production] |
| 19:50 |
<cmooney@cumin1002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 19:50 |
<cmooney@cumin1002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: reset dns names for cloudcontrol1011 to newly-assigned ones - cmooney@cumin1002" |
[production] |
| 19:50 |
<cmooney@cumin1002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: reset dns names for cloudcontrol1011 to newly-assigned ones - cmooney@cumin1002" |
[production] |
| 19:41 |
<cmooney@cumin1002> |
START - Cookbook sre.dns.netbox |
[production] |