3601-3650 of 10000 results (21ms)
2020-11-18 ยง
21:02 <mutante> mw1317,mw1318 - repooled=no after physical move to rack B [production]
20:56 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1318.eqiad.wmnet [production]
20:54 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1317.eqiad.wmnet [production]
20:27 <mutante> mw1317, mw1318 shutting down for physical move [production]
20:21 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw1318.eqiad.wmnet [production]
20:21 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw1317.eqiad.wmnet [production]
20:15 <mutante> mw1317,mw1318 - downtimed and depooled - they are physically moving from B7 to B5 (T266164) [production]
20:13 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:13 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:13 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:13 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:11 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1317.eqiad.wmnet [production]
20:11 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw1318.eqiad.wmnet [production]
20:10 <dancy@deploy1001> Synchronized php: group1 wikis to 1.36.0-wmf.18 (duration: 01m 03s) [production]
20:09 <dancy@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.18 [production]
20:03 <akosiaris@cumin1001> conftool action : set/pooled=inactive; selector: service=recommendation-api,name=kubernetes.*,dc=codfw [production]
20:03 <akosiaris@cumin1001> conftool action : set/weight=0; selector: service=recommendation-api,name=kubernetes.*,dc=codfw [production]
19:53 <akosiaris@cumin1001> conftool action : set/pooled=no; selector: service=recommendation-api,name=kubernetes.*,dc=codfw [production]
19:48 <otto@deploy1001> Synchronized php-1.36.0-wmf.16/extensions/EventLogging/modules/ext.eventLogging/core.js: EventLogging legacy events should use dt as server side receive time - T240460 (duration: 01m 06s) [production]
19:45 <otto@deploy1001> Synchronized php-1.36.0-wmf.18/extensions/EventLogging/modules/ext.eventLogging/core.js: EventLogging legacy events should use dt as server side receive time - T240460 (duration: 01m 07s) [production]
19:26 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:23 <ppchelko@deploy1001> Synchronized wmf-config/InitialiseSettings.php: gerrit:635607 - Switch ParserCache to JSON for group0 wikis (duration: 01m 05s) [production]
19:19 <ppchelko@deploy1001> Synchronized wmf-config/CommonSettings.php: gerrit:635086 - Enable parsoid on api_appserver (duration: 01m 04s) [production]
19:19 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
19:13 <ppchelko@deploy1001> Synchronized wmf-config/CommonSettings.php: gerrit:641527 - Set to 0 (duration: 01m 04s) [production]
18:26 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
18:26 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:44 <jayme@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:38 <jayme@cumin1001> START - Cookbook sre.dns.netbox [production]
17:18 <elukey> shutdown an-presto1004 for hw maintenance [production]
17:13 <akosiaris> T241230 pool codfw kubernetes for recommendation-api at a very low weight [production]
17:12 <akosiaris@cumin1001> conftool action : set/pooled=yes; selector: service=recommendation-api,name=kubernetes.*,dc=codfw [production]
17:12 <akosiaris@cumin1001> conftool action : set/weight=1; selector: service=recommendation-api,name=kubernetes.*,dc=codfw [production]
16:52 <jbond42> drop os_version/requiers_os functions from wmflib [production]
16:50 <elukey> update /etc/krb5.keytab on krb1001/krb2001 to match the most up to date key version for host/krb2001.codfw.wmnet [production]
16:49 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
16:49 <hnowlan@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
16:44 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
16:43 <hnowlan@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
16:38 <reedy@deploy1001> Synchronized wmf-config/logging.php: T268141 (duration: 01m 06s) [production]
16:36 <pt1979@cumin2001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:32 <robh@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:31 <pt1979@cumin2001> START - Cookbook sre.dns.netbox [production]
16:27 <robh@cumin1001> START - Cookbook sre.dns.netbox [production]
15:59 <akosiaris@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . [production]
15:56 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . [production]
15:51 <akosiaris@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . [production]
15:42 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
15:16 <Urbanecm> mwscript deleteEqualMessages.php --wiki=cswiki --delete [production]
15:14 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'recommendation-api' for release 'production' . [production]