1-50 of 10000 results (64ms)
2023-10-05 ยง
23:22 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1003.eqiad.wmnet with OS bullseye [production]
23:22 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-master1003.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:19 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host an-master1003.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:02 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bullseye [production]
23:02 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-master1004.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:00 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host an-master1004.mgmt.eqiad.wmnet with reboot policy FORCED [production]
22:59 <pt1979@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1007.eqiad.wmnet with OS bullseye [production]
22:58 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host an-master1004.mgmt.eqiad.wmnet with reboot policy FORCED [production]
22:58 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host an-master1004.mgmt.eqiad.wmnet with reboot policy FORCED [production]
22:37 <pt1979@cumin1001> START - Cookbook sre.hosts.reimage for host cloudelastic1007.eqiad.wmnet with OS bullseye [production]
21:34 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1003.eqiad.wmnet with OS bullseye [production]
21:32 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1004.eqiad.wmnet with OS bullseye [production]
21:17 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching restbase2020.codfw.wmnet: Maybe cleanup leaked file descriptors(?) - eevans@cumin1001 [production]
21:07 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching restbase2020.codfw.wmnet: Maybe cleanup leaked file descriptors(?) - eevans@cumin1001 [production]
21:03 <thcipriani@deploy2002> Finished scap: Backport for [[gerrit:962082|[foundationwiki] Add Endowment, Agenda, Committee, and Memory namespaces (T347762 T347822 T348268)]], [[gerrit:963801|[foundationwiki] Provide 'translationadmin' group with 'edit-legal' right (T346187)]] (duration: 09m 56s) [production]
20:58 <thcipriani@deploy2002> thcipriani and varnent: Continuing with sync [production]
20:57 <eileen> civicrm upgraded from 05545fbc to 4329014b [production]
20:55 <thcipriani@deploy2002> thcipriani and varnent: Backport for [[gerrit:962082|[foundationwiki] Add Endowment, Agenda, Committee, and Memory namespaces (T347762 T347822 T348268)]], [[gerrit:963801|[foundationwiki] Provide 'translationadmin' group with 'edit-legal' right (T346187)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:53 <thcipriani@deploy2002> Started scap: Backport for [[gerrit:962082|[foundationwiki] Add Endowment, Agenda, Committee, and Memory namespaces (T347762 T347822 T348268)]], [[gerrit:963801|[foundationwiki] Provide 'translationadmin' group with 'edit-legal' right (T346187)]] [production]
20:49 <thcipriani@deploy2002> Finished scap: Backport for [[gerrit:963810|[Prototype] Add screen resolution to Typography prototype]], [[gerrit:963811|[Prototype] Edit project link page on reading prototype]] (duration: 23m 57s) [production]
20:39 <thcipriani@deploy2002> jdrewniak and thcipriani: Continuing with sync [production]
20:37 <thcipriani@deploy2002> jdrewniak and thcipriani: Backport for [[gerrit:963810|[Prototype] Add screen resolution to Typography prototype]], [[gerrit:963811|[Prototype] Edit project link page on reading prototype]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:25 <thcipriani@deploy2002> Started scap: Backport for [[gerrit:963810|[Prototype] Add screen resolution to Typography prototype]], [[gerrit:963811|[Prototype] Edit project link page on reading prototype]] [production]
20:22 <thcipriani@deploy2002> Finished scap: Backport for [[gerrit:963799|Enable Minerva site notice for Nepali Wikipedia (newiki) (T347814)]] (duration: 08m 57s) [production]
20:16 <thcipriani@deploy2002> ammarpad and thcipriani: Continuing with sync [production]
20:14 <thcipriani@deploy2002> ammarpad and thcipriani: Backport for [[gerrit:963799|Enable Minerva site notice for Nepali Wikipedia (newiki) (T347814)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:14 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1003.eqiad.wmnet with OS bullseye [production]
20:14 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host an-master1004.eqiad.wmnet with OS bullseye [production]
20:13 <thcipriani@deploy2002> Started scap: Backport for [[gerrit:963799|Enable Minerva site notice for Nepali Wikipedia (newiki) (T347814)]] [production]
18:51 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host ganeti-test2004.codfw.wmnet with OS bullseye [production]
18:47 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ganeti-test2004'] [production]
18:47 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ganeti-test2004'] [production]
18:45 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:43 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
18:34 <jhuneidi@deploy2002> rebuilt and synchronized wikiversions files: group2 wikis to 1.41.0-wmf.29 refs T347080 [production]
18:17 <sukhe> running authdns-update: T347054 [production]
18:15 <jhuneidi@deploy2002> Synchronized php: group1 wikis to 1.41.0-wmf.29 refs T347080 (duration: 06m 12s) [production]
18:08 <jhuneidi@deploy2002> rebuilt and synchronized wikiversions files: group1 wikis to 1.41.0-wmf.29 refs T347080 [production]
17:39 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ganeti-test2004'] [production]
17:39 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ganeti-test2004'] [production]
17:35 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti-test2004.mgmt.codfw.wmnet with reboot policy FORCED [production]
17:30 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host ganeti-test2004.mgmt.codfw.wmnet with reboot policy FORCED [production]
17:29 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti-test2004.mgmt.codfw.wmnet with reboot policy FORCED [production]
17:27 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host ganeti-test2004.mgmt.codfw.wmnet with reboot policy FORCED [production]
17:26 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:25 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
16:57 <bvibber> scaling back batch jobs for T312153 and T312152, will run these in further chunks as the new config rolls out [production]
16:47 <bvibber> brion running requeueTranscodes.php on mwmaint2002 for VP9 transcode cleanup for T312153 [production]
16:22 <volans> installed 7.3.1 on cumin1001 [production]
16:19 <jbond@cumin2002> END (PASS) - Cookbook sre.puppetboard.restart-reboot (exit_code=0) rolling reboot on A:puppetboard [production]