4251-4300 of 10000 results (31ms)
2023-04-04 §
12:28 <volans@cumin1001> START - Cookbook sre.netbox.update-extras rolling update on A:netbox [production]
12:28 <volans@cumin1001> END (FAIL) - Cookbook sre.netbox.update-extras (exit_code=1) rolling update on A:netbox-canary [production]
12:27 <volans@cumin1001> START - Cookbook sre.netbox.update-extras rolling update on A:netbox-canary [production]
07:27 <hashar@deploy2002> Finished deploy [gerrit/gerrit@453b038]: Gerrit plugin update and switching from git-fat to git-lfs (duration: 00m 08s) [production]
07:27 <hashar@deploy2002> Started deploy [gerrit/gerrit@453b038]: Gerrit plugin update and switching from git-fat to git-lfs [production]
07:23 <hashar@deploy2002> Finished deploy [gerrit/gerrit@453b038]: Gerrit plugin update and switching from git-fat to git-lfs (duration: 00m 05s) [production]
07:23 <hashar@deploy2002> Started deploy [gerrit/gerrit@453b038]: Gerrit plugin update and switching from git-fat to git-lfs [production]
2023-04-03 §
15:52 <jdrewniak@deploy2002> Synchronized portals: Wikimedia Portals Update: [[gerrit:905247| Bumping portals to master (T128546)]] (duration: 05m 33s) [production]
15:46 <jdrewniak@deploy2002> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:905247| Bumping portals to master (T128546)]] (duration: 06m 14s) [production]
2023-03-30 §
11:50 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update dns an-worker1149-56 - jclark@cumin1001" [production]
11:49 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update dns an-worker1149-56 - jclark@cumin1001" [production]
2023-03-29 §
20:15 <taavi@deploy2002> Finished scap: Backport for [[gerrit:903835|Update "United States" static page to facilitate synthetic testing of T331681 (T331681)]] (duration: 09m 45s) [production]
20:07 <taavi@deploy2002> nray and taavi: Backport for [[gerrit:903835|Update "United States" static page to facilitate synthetic testing of T331681 (T331681)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
20:05 <taavi@deploy2002> Started scap: Backport for [[gerrit:903835|Update "United States" static page to facilitate synthetic testing of T331681 (T331681)]] [production]
10:58 <claime> authdns-update successful on all nodes - T333120 [production]
10:57 <claime> Running authdns-update [production]
07:07 <slyngs> Update Squid logformat [production]
06:43 <hashar@deploy2002> Finished deploy [gerrit/gerrit@e7c1696]: Update Gerrit javascript plugins (duration: 00m 10s) [production]
06:43 <hashar@deploy2002> Started deploy [gerrit/gerrit@e7c1696]: Update Gerrit javascript plugins [production]
06:40 <hashar@deploy2002> Finished deploy [gerrit/gerrit@e7c1696]: Update Gerrit javascript plugins (duration: 00m 06s) [production]
06:40 <hashar@deploy2002> Started deploy [gerrit/gerrit@e7c1696]: Update Gerrit javascript plugins [production]
2023-03-28 §
14:31 <sukhe> run authdns-update to revert eqiad depool [production]
08:48 <AndyRussG> update payments.wiki config 65bedd4a -> e31ffd7d, payments (automatic updates only) a6c6c2b1 -> f5ec2677 [production]
05:47 <AndyRussG> update payments-wiki f5e262d1 -> a6c6c2b1 [production]
2023-03-24 §
10:55 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 21 days, 0:00:00 on krb2002.codfw.wmnet with reason: Non-functional, WIP for Bullseye update [production]
10:55 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 21 days, 0:00:00 on krb2002.codfw.wmnet with reason: Non-functional, WIP for Bullseye update [production]
05:26 <hashar@deploy2002> Finished deploy [gerrit/gerrit@c1cbda4]: Update js plugins for EarlyWarning bot (T330850) and displaying Zuul status on changes (T241068) (duration: 00m 10s) [production]
05:25 <hashar@deploy2002> Started deploy [gerrit/gerrit@c1cbda4]: Update js plugins for EarlyWarning bot (T330850) and displaying Zuul status on changes (T241068) [production]
05:21 <hashar@deploy2002> Finished deploy [gerrit/gerrit@c1cbda4]: Update js plugins for EarlyWarning bot (T330850) and displaying Zuul status on changes (T241068) (duration: 00m 07s) [production]
05:20 <hashar@deploy2002> Started deploy [gerrit/gerrit@c1cbda4]: Update js plugins for EarlyWarning bot (T330850) and displaying Zuul status on changes (T241068) [production]
2023-03-22 §
15:23 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-main2004.codfw.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
15:23 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-main2004.codfw.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
11:16 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on kafka-main2005.codfw.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
11:16 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on kafka-main2005.codfw.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
10:23 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on kafka-main2005.codfw.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
10:22 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on kafka-main2005.codfw.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
09:01 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on kafka-main1004.eqiad.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
09:01 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 3:00:00 on kafka-main1004.eqiad.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
00:57 <zabe@deploy2002> Finished scap: update interwiki cache (duration: 07m 02s) [production]
00:50 <zabe@deploy2002> Started scap: update interwiki cache [production]
2023-03-21 §
21:02 <AndyRussG> update SmashPig config 6e651fd4 -> 035f602a [production]
18:00 <AndyRussG> update SmashPig config 59a8b2d2 -> 6e651fd [production]
13:11 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on kafka-main1005.eqiad.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
13:11 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 3:00:00 on kafka-main1005.eqiad.wmnet with reason: Stop kafka, update idrac/bios/nic-firmware [production]
2023-03-20 §
17:55 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:40:00 on lvs1019.eqiad.wmnet with reason: reboot for kernel update [production]
17:55 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 0:40:00 on lvs1019.eqiad.wmnet with reason: reboot for kernel update [production]
17:14 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:40:00 on lvs3006.esams.wmnet with reason: reboot for kernel update [production]
17:14 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 0:40:00 on lvs3006.esams.wmnet with reason: reboot for kernel update [production]
17:14 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:40:00 on lvs2009.codfw.wmnet,lvs1019.eqiad.wmnet with reason: reboot for kernel update [production]
17:14 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 0:40:00 on lvs2009.codfw.wmnet,lvs1019.eqiad.wmnet with reason: reboot for kernel update [production]