1801-1850 of 10000 results (28ms)
2023-05-06 §
06:50 <jelto@cumin1001> START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: Install software version upgrade [production]
2023-05-05 §
23:39 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/916634 [releng]
23:24 <tzatziki> removing emails from 230 users per self-requests [production]
22:21 <bd808> Added "RepoLookoutBot" to hiera key "dynamicproxy::blocked_user_agent_regex" to stop unnecessary scans by https://www.repo-lookout.org/ [tools]
22:20 <bd808> Added [tools]
18:57 <brennen@deploy1002> Finished scap: Backport for [[gerrit:915719|Revert "api: Use RevisionStore::newRevisionsFromBatch to fetch revision records" (T336008 T336022)]] (duration: 14m 21s) [production]
18:44 <brennen@deploy1002> umherirrender and brennen: Backport for [[gerrit:915719|Revert "api: Use RevisionStore::newRevisionsFromBatch to fetch revision records" (T336008 T336022)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet [production]
18:42 <brennen@deploy1002> Started scap: Backport for [[gerrit:915719|Revert "api: Use RevisionStore::newRevisionsFromBatch to fetch revision records" (T336008 T336022)]] [production]
18:25 <brennen> train 1.41.0-wmf.7 (T330213): trying revert for T336008, T336022 [production]
17:45 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/916527 [releng]
17:42 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs2011.codfw.wmnet with OS bullseye [production]
17:27 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs2011.codfw.wmnet with reason: host reimage [production]
17:24 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on lvs2011.codfw.wmnet with reason: host reimage [production]
17:09 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]
17:08 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs2011.codfw.wmnet with OS bullseye [production]
16:53 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]
16:52 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs2011.codfw.wmnet with OS bullseye [production]
16:52 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]
16:45 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]
16:45 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs2011.codfw.wmnet with OS bullseye [production]
16:44 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]
16:44 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on lvs2011.codfw.wmnet with reason: host reimage [production]
16:35 <btullis@cumin1001> END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) [production]
16:35 <btullis@cumin1001> Added views for new wiki: newiki T334041 [production]
16:29 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]
16:28 <pt1979@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host lvs2011.codfw.wmnet with OS bullseye [production]
16:20 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1023.eqiad.wmnet [production]
16:20 <andrew@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:20 <andrew@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1023.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
16:18 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1024.eqiad.wmnet [production]
16:17 <andrew@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:17 <andrew@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1023.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
16:16 <andrew@cumin1001> START - Cookbook sre.dns.netbox [production]
16:15 <andrew@cumin1001> START - Cookbook sre.dns.netbox [production]
16:13 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]
16:10 <btullis@cumin1001> START - Cookbook sre.wikireplicas.add-wiki [production]
16:10 <andrew@cumin1001> START - Cookbook sre.hosts.decommission for hosts cloudvirt1024.eqiad.wmnet [production]
16:10 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs2011.codfw.wmnet with OS bullseye [production]
16:10 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]
16:08 <andrew@cumin1001> START - Cookbook sre.hosts.decommission for hosts cloudvirt1023.eqiad.wmnet [production]
16:07 <wm-bot2> Drained cloudvirt1024.eqiad.wmnet (T336064) - cookbook ran by andrew@bullseye [admin]
16:06 <btullis@cumin1001> END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) [production]
16:06 <btullis@cumin1001> Added views for new wiki: zhwiki T334041 [production]
16:03 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs2011.codfw.wmnet with OS bullseye [production]
16:03 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]
16:03 <wm-bot2> Set cloudvirt cloudvirt1024.eqiad.wmnet maintenance (downtime id: 95995009-09d6-496e-8cd2-0cfac93d3cf7, use this to unset) (T336064) - cookbook ran by andrew@bullseye [admin]
16:02 <wm-bot2> Draining cloudvirt1024.eqiad.wmnet (T336064) - cookbook ran by andrew@bullseye [admin]
16:01 <wm-bot2> Drained cloudvirt1023.eqiad.wmnet (T336064) - cookbook ran by andrew@bullseye [admin]
16:00 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs2011.codfw.wmnet with OS bullseye [production]
16:00 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2011.codfw.wmnet with OS bullseye [production]