3501-3550 of 10000 results (35ms)
2020-05-25 §
09:17 <_joe_> migrated mw1337 to use envoy for TLS termination T247389 [production]
09:10 <jayme@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'zotero' for release 'staging' . [production]
09:04 <godog> turn on sni by default for check_http --ssl icinga invocations - T253292 [production]
08:52 <jayme@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
08:39 <jayme@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
08:21 <filippo@cumin1001> conftool action : set/pooled=yes:weight=100; selector: service=thanos-swift [production]
08:05 <jayme@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'termbox' for release 'staging' . [production]
07:36 <moritzm> installed linux-image-amd64 on labstore1005 (current meta package for kernels following the Stretch update) T224582 [production]
07:36 <moritzm> installed linux-imageamd64 on labstore (current meta package for kernels following the Stretch update) T224582 [production]
07:02 <marostegui> Stop event scheduler on tendril T252331 [production]
05:11 <marostegui> Deploy schema change on s6, directly on the master - T253342 [production]
04:54 <marostegui> Depool labsdb1011 - T249188 [production]
04:11 <kart_> Updated cxserver to 2020-05-22-083137-production (T246317, T252871) [production]
04:07 <kartik@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
04:04 <kartik@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
04:02 <kartik@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'cxserver' for release 'staging' . [production]
2020-05-24 §
17:36 <gehel> restarting elasticsearch psi on elastic1052 [production]
16:44 <gehel> depool wdqs1007 to catch on lag [production]
16:43 <gehel> restart blazegraph on wdqs1007 [production]
2020-05-23 §
19:04 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.31/includes/filerepo/file/LocalFile.php: I0f7e885997d60 (duration: 01m 06s) [production]
18:58 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.32/includes/filerepo/file/LocalFile.php: I0f7e885997d60 (duration: 01m 08s) [production]
18:06 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.32/includes/filerepo/: I31a9bb6672 (duration: 01m 06s) [production]
18:05 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.31/includes/filerepo/: I31a9bb6672 (duration: 01m 10s) [production]
15:44 <krinkle@deploy1001> Synchronized wmf-config/mc.php: I5ad8fe96b9098a8 - Disable coalesceKeys on commonswiki (duration: 01m 09s) [production]
14:58 <Krinkle> scap-pull to reset state on mwdebug1002 [production]
14:50 <Krinkle> Testing mc.php changes on mwdebug1002 [production]
08:04 <elukey> powercycle an-presto1004 - unresponsive, racadm getsel shows CPU overheating alerts [production]
2020-05-22 §
22:42 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.31/includes/filerepo/: Ie19613ef7643a (duration: 01m 06s) [production]
22:40 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.32/includes/filerepo/: Ie19613ef7643a (duration: 01m 08s) [production]
15:58 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
15:58 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
15:57 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
15:53 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
15:47 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
15:45 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
15:45 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
15:30 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
15:25 <cdanis> fixing prometheus-nic-firmware-textfile.service wherever it is broken T253374 [production]
15:25 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . [production]
15:24 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . [production]
15:06 <marostegui> Decrease tendril_purge_global_status_log_5m storing rows time from 2 days to 1 day T252331 [production]
15:01 <kormat@cumin1001> dbctl commit (dc=all): 'Pool db2137 into s4+s5 T252985', diff saved to https://phabricator.wikimedia.org/P11292 and previous config saved to /var/cache/conftool/dbconfig/20200522-150120-kormat.json [production]
14:53 <reedy@deploy1001> Synchronized php-1.35.0-wmf.31/maintenance/blockUsers.php: (no justification provided) (duration: 01m 08s) [production]
14:51 <reedy@deploy1001> Synchronized php-1.35.0-wmf.32/maintenance/blockUsers.php: (no justification provided) (duration: 01m 09s) [production]
14:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1144:331[45] and db1097:331[45]', diff saved to https://phabricator.wikimedia.org/P11290 and previous config saved to /var/cache/conftool/dbconfig/20200522-143541-marostegui.json [production]
14:24 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
14:22 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
14:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1144:331[45] and db1097:331[45]', diff saved to https://phabricator.wikimedia.org/P11289 and previous config saved to /var/cache/conftool/dbconfig/20200522-141513-marostegui.json [production]
14:13 <sukhe> upload dnsdist_1.4.0-1~deb10u1 to apt.wm.o (buster) - T252132 [production]
14:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1144:331[45] and db1097:331[45]', diff saved to https://phabricator.wikimedia.org/P11288 and previous config saved to /var/cache/conftool/dbconfig/20200522-140847-marostegui.json [production]