|
2020-05-25
§
|
| 09:49 |
<_joe_> |
depooled mw1337, it was getting all traffic supposed to go to the jobrunners |
[production] |
| 09:45 |
<vgutierrez> |
upload trafficserver 8.0.7-1wm10 to apt.wm.o (buster) |
[production] |
| 09:42 |
<_joe_> |
converting mw1319-1333 to use envoy for TLS termination |
[production] |
| 09:17 |
<_joe_> |
migrated mw1337 to use envoy for TLS termination T247389 |
[production] |
| 09:10 |
<jayme@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'zotero' for release 'staging' . |
[production] |
| 09:04 |
<godog> |
turn on sni by default for check_http --ssl icinga invocations - T253292 |
[production] |
| 08:52 |
<jayme@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 08:39 |
<jayme@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'termbox' for release 'production' . |
[production] |
| 08:21 |
<filippo@cumin1001> |
conftool action : set/pooled=yes:weight=100; selector: service=thanos-swift |
[production] |
| 08:05 |
<jayme@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'termbox' for release 'staging' . |
[production] |
| 07:36 |
<moritzm> |
installed linux-image-amd64 on labstore1005 (current meta package for kernels following the Stretch update) T224582 |
[production] |
| 07:36 |
<moritzm> |
installed linux-imageamd64 on labstore (current meta package for kernels following the Stretch update) T224582 |
[production] |
| 07:02 |
<marostegui> |
Stop event scheduler on tendril T252331 |
[production] |
| 05:11 |
<marostegui> |
Deploy schema change on s6, directly on the master - T253342 |
[production] |
| 04:54 |
<marostegui> |
Depool labsdb1011 - T249188 |
[production] |
| 04:11 |
<kart_> |
Updated cxserver to 2020-05-22-083137-production (T246317, T252871) |
[production] |
| 04:07 |
<kartik@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'cxserver' for release 'production' . |
[production] |
| 04:04 |
<kartik@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'cxserver' for release 'production' . |
[production] |
| 04:02 |
<kartik@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'cxserver' for release 'staging' . |
[production] |
|
2020-05-23
§
|
| 19:04 |
<krinkle@deploy1001> |
Synchronized php-1.35.0-wmf.31/includes/filerepo/file/LocalFile.php: I0f7e885997d60 (duration: 01m 06s) |
[production] |
| 18:58 |
<krinkle@deploy1001> |
Synchronized php-1.35.0-wmf.32/includes/filerepo/file/LocalFile.php: I0f7e885997d60 (duration: 01m 08s) |
[production] |
| 18:06 |
<krinkle@deploy1001> |
Synchronized php-1.35.0-wmf.32/includes/filerepo/: I31a9bb6672 (duration: 01m 06s) |
[production] |
| 18:05 |
<krinkle@deploy1001> |
Synchronized php-1.35.0-wmf.31/includes/filerepo/: I31a9bb6672 (duration: 01m 10s) |
[production] |
| 15:44 |
<krinkle@deploy1001> |
Synchronized wmf-config/mc.php: I5ad8fe96b9098a8 - Disable coalesceKeys on commonswiki (duration: 01m 09s) |
[production] |
| 14:58 |
<Krinkle> |
scap-pull to reset state on mwdebug1002 |
[production] |
| 14:50 |
<Krinkle> |
Testing mc.php changes on mwdebug1002 |
[production] |
| 08:04 |
<elukey> |
powercycle an-presto1004 - unresponsive, racadm getsel shows CPU overheating alerts |
[production] |
|
2020-05-22
§
|
| 22:42 |
<krinkle@deploy1001> |
Synchronized php-1.35.0-wmf.31/includes/filerepo/: Ie19613ef7643a (duration: 01m 06s) |
[production] |
| 22:40 |
<krinkle@deploy1001> |
Synchronized php-1.35.0-wmf.32/includes/filerepo/: Ie19613ef7643a (duration: 01m 08s) |
[production] |
| 15:58 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:58 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:57 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:53 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:47 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:45 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:45 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:30 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:25 |
<cdanis> |
fixing prometheus-nic-firmware-textfile.service wherever it is broken T253374 |
[production] |
| 15:25 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:24 |
<hnowlan@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . |
[production] |
| 15:06 |
<marostegui> |
Decrease tendril_purge_global_status_log_5m storing rows time from 2 days to 1 day T252331 |
[production] |
| 15:01 |
<kormat@cumin1001> |
dbctl commit (dc=all): 'Pool db2137 into s4+s5 T252985', diff saved to https://phabricator.wikimedia.org/P11292 and previous config saved to /var/cache/conftool/dbconfig/20200522-150120-kormat.json |
[production] |
| 14:53 |
<reedy@deploy1001> |
Synchronized php-1.35.0-wmf.31/maintenance/blockUsers.php: (no justification provided) (duration: 01m 08s) |
[production] |
| 14:51 |
<reedy@deploy1001> |
Synchronized php-1.35.0-wmf.32/maintenance/blockUsers.php: (no justification provided) (duration: 01m 09s) |
[production] |
| 14:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1144:331[45] and db1097:331[45]', diff saved to https://phabricator.wikimedia.org/P11290 and previous config saved to /var/cache/conftool/dbconfig/20200522-143541-marostegui.json |
[production] |
| 14:24 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 14:22 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |