3101-3150 of 10000 results (72ms)
2022-08-11 ยง
13:36 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:36 <awight@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:822130|trwikiquote: Install WikiLove extension (T314895)]] (duration: 03m 30s) [production]
13:35 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:35 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:34 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:33 <filippo@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host logstash2003.codfw.wmnet [production]
13:25 <awight@deploy1002> Synchronized static/images: Config: [[gerrit:821330|Revert "trwiki: Change old and new vector logos for 500k articles"]] (part 3) (duration: 03m 09s) [production]
13:21 <awight@deploy1002> Synchronized logos/: Config: [[gerrit:821330|Revert "trwiki: Change old and new vector logos for 500k articles"]] (part 2) (duration: 03m 09s) [production]
13:19 <topranks> merging CR821781 to expose additional network info in puppet facts [production]
13:18 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:18 <awight@deploy1002> Synchronized wmf-config/: Config: [[gerrit:821330|Revert "trwiki: Change old and new vector logos for 500k articles"]] (part 1) (duration: 03m 13s) [production]
13:17 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:17 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:16 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:14 <ryankemper@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on elastic1060.eqiad.wmnet with reason: host reimage [production]
13:11 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:11 <ryankemper@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on elastic1060.eqiad.wmnet with reason: host reimage [production]
13:10 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:09 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:08 <awight@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:822073|Enable editor line numbering on all namespaces, for twwiki (T302852)]] (duration: 03m 42s) [production]
12:56 <ryankemper@cumin1001> START - Cookbook sre.hosts.reimage for host elastic1060.eqiad.wmnet with OS bullseye [production]
12:55 <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.REIMAGE (1 nodes at a time) for ElasticSearch cluster search_eqiad: eqiad cluster reimage (bullseye upgrade) - ryankemper@cumin1001 - T289135 [production]
12:49 <aikochou@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
12:46 <aikochou@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
12:26 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=restbase2018.codfw.wmnet [production]
12:26 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=restbase202[367].codfw.wmnet [production]
12:17 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
12:17 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
12:17 <elukey@deploy1002> helmfile [ml-serve-codfw] DONE helmfile.d/admin 'sync'. [production]
12:16 <elukey@deploy1002> helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. [production]
12:13 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host logstash2003.codfw.wmnet [production]
12:11 <elukey@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
12:10 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
12:09 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
11:20 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
11:20 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
09:57 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
09:57 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
09:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
09:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
09:49 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
09:48 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
09:32 <godog> arm keyholder on netmon2001 [production]
09:09 <jbond> update gnutls28 on bullseye systems [production]
09:00 <jbond> update unzip [production]
08:21 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
08:13 <elukey@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
08:12 <elukey@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
08:06 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.debug (exit_code=0) for Netbox interface ID cr3-ulsfo:xe-0/1/1 [production]