4501-4550 of 10000 results (84ms)
2023-07-26 §
06:25 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
06:24 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
06:21 <denisse@cumin1001> START - Cookbook sre.hosts.decommission for hosts xhgui2001.codfw.wmnet,xhgui1001.eqiad.wmnet [production]
06:18 <oblivian@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
06:17 <oblivian@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
06:17 <oblivian@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
06:15 <oblivian@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
01:07 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host rdb1013.eqiad.wmnet with OS bullseye [production]
2023-07-25 §
22:52 <eileen> revision c62433ab -> 8689d10d [production]
21:18 <zabe@deploy1002> Finished scap: Backport for [[gerrit:941498|Create UserIdentityValue with correct wiki (T342655)]], [[gerrit:941499|Create UserIdentityValue with correct wiki (T342655)]] (duration: 10m 06s) [production]
21:10 <zabe@deploy1002> zabe: Backport for [[gerrit:941498|Create UserIdentityValue with correct wiki (T342655)]], [[gerrit:941499|Create UserIdentityValue with correct wiki (T342655)]] synced to the testservers mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
21:08 <zabe@deploy1002> Started scap: Backport for [[gerrit:941498|Create UserIdentityValue with correct wiki (T342655)]], [[gerrit:941499|Create UserIdentityValue with correct wiki (T342655)]] [production]
21:02 <zabe@deploy1002> Finished scap: update interwiki cache, [[gerrit:941057]] (duration: 07m 20s) [production]
20:55 <zabe@deploy1002> Started scap: update interwiki cache, [[gerrit:941057]] [production]
20:53 <zabe@deploy1002> Finished scap: T335216 (duration: 08m 24s) [production]
20:46 <zabe@deploy1002> zabe: T335216 synced to the testservers mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
20:44 <zabe@deploy1002> Started scap: T335216 [production]
20:42 <zabe> create Wiktionary Mandailing # T335216 [production]
20:33 <taavi@deploy1002> Finished scap: Backport for [[gerrit:941423|Fix text showing on icon only buttons]] (duration: 12m 08s) [production]
20:23 <taavi@deploy1002> taavi and bwang: Backport for [[gerrit:941423|Fix text showing on icon only buttons]] synced to the testservers mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
20:21 <taavi@deploy1002> Started scap: Backport for [[gerrit:941423|Fix text showing on icon only buttons]] [production]
18:24 <dwisehaupt@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:24 <dwisehaupt@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: * - dwisehaupt@cumin1001" [production]
18:23 <dwisehaupt@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: * - dwisehaupt@cumin1001" [production]
18:21 <dwisehaupt@cumin1001> START - Cookbook sre.dns.netbox [production]
18:21 <sukhe> dummy authdns-update returns [production]
18:11 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dns4003.wikimedia.org [production]
18:06 <sukhe@cumin2002> START - Cookbook sre.hosts.reboot-single for host dns4003.wikimedia.org [production]
17:56 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dns4004.wikimedia.org [production]
17:51 <sukhe@cumin2002> START - Cookbook sre.hosts.reboot-single for host dns4004.wikimedia.org [production]
16:56 <sukhe> dummy authdns-update [production]
16:51 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/thumbor: apply [production]
16:51 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/thumbor: apply [production]
16:49 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dns6002.wikimedia.org [production]
16:45 <sukhe@cumin2002> START - Cookbook sre.hosts.reboot-single for host dns6002.wikimedia.org [production]
16:41 <fabfur> end rebooting lvs5005 (T335835) [production]
16:41 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host lvs5005.eqsin.wmnet [production]
16:40 <elukey@cumin1001> END (PASS) - Cookbook sre.kafka.roll-restart-brokers (exit_code=0) for Kafka A:kafka-main-eqiad cluster: Roll restart of jvm daemons. [production]
16:37 <fabfur@cumin1001> START - Cookbook sre.hosts.reboot-single for host lvs5005.eqsin.wmnet [production]
16:30 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dns6001.wikimedia.org [production]
16:26 <sukhe@cumin2002> START - Cookbook sre.hosts.reboot-single for host dns6001.wikimedia.org [production]
16:19 <fabfur> begin rebooting lvs5005 (T335835) [production]
15:57 <dancy@deploy1002> Finished deploy [releng/jenkins-deploy@97b4674] (releasing): (no justification provided) (duration: 33m 26s) [production]
15:36 <fabfur> lvs5004 restarted and services are reactivating (T335835) [production]
15:28 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host lvs5004.eqsin.wmnet [production]
15:26 <isaranto@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
15:25 <fabfur@cumin1001> START - Cookbook sre.hosts.reboot-single for host lvs5004.eqsin.wmnet [production]
15:25 <fabfur@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host lvs5004.eqsin.wmnet [production]
15:25 <fabfur@cumin1001> START - Cookbook sre.hosts.reboot-single for host lvs5004.eqsin.wmnet [production]
15:25 <isaranto@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]