|
2019-07-12
§
|
| 16:32 |
<urandom> |
bootstrapping restbase1017-a -- T222960 |
[production] |
| 16:25 |
<jijiki> |
Rolling restart swift proxy on ms-fe* |
[production] |
| 15:25 |
<jeh> |
rebooting cloudvirt1018.eqiad.wmnet T216040 |
[production] |
| 14:05 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 12:45 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 12:39 |
<fsero> |
recreating ci staging namespaces T227775 |
[production] |
| 12:39 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'blubberoid' for release 'staging' . |
[production] |
| 12:38 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'blubberoid' for release 'staging' . |
[production] |
| 12:36 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-main' for release 'main' . |
[production] |
| 12:33 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . |
[production] |
| 12:33 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . |
[production] |
| 12:22 |
<fsero> |
recreating eventgate-* and blubberoid staging namespaces T227775 |
[production] |
| 12:22 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'mathoid' for release 'staging' . |
[production] |
| 12:22 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'mathoid' for release 'staging' . |
[production] |
| 12:18 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 12:18 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 12:18 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 12:15 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'sessionstore' for release 'staging' . |
[production] |
| 12:11 |
<fsero> |
recreating sessionstore,cxserver and mathoid staging namespaces T227775 |
[production] |
| 12:10 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'citoid' for release 'staging' . |
[production] |
| 12:06 |
<fsero> |
recreating citoid staging namespace T227775 |
[production] |
| 12:05 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'termbox' for release 'staging' . |
[production] |
| 12:01 |
<fsero> |
recreating termbox staging namespace T227775 |
[production] |
| 11:09 |
<jynus@deploy1001> |
Synchronized wmf-config/db-codfw.php: Switchover db2045 x1 codfw master to db2069 (duration: 00m 51s) |
[production] |
| 10:24 |
<jynus> |
switchover x1 codfw master from db2045 to db2069 T227862 |
[production] |
| 10:23 |
<jynus> |
switchover x1 codfw master from db2045 to db2069 |
[production] |
| 09:43 |
<moritzm> |
shut down ldap-codfw-replica01/ldap-codfw-replica02 (pending reimage) |
[production] |
| 08:18 |
<jijiki> |
enable puppet on mw1222 |
[production] |
| 06:35 |
<vgutierrez> |
upgrading acme-chief to version 0.19 in acme-chief test instances - T225945 |
[production] |
| 06:28 |
<vgutierrez> |
uploaded acme-chief 0.19 to apt.wikimedia.org (buster) - T225945 |
[production] |
| 05:45 |
<elukey> |
sudo -i /usr/local/sbin/restart-php7.2-fpm on mwdebug* to clear opcache |
[production] |
| 01:01 |
<Krinkle> |
mw1342 generated some ~ 11,500 additional PHP errors over a 4 hour period (18:00-22:30 UTC), ref T224491 |
[production] |
| 00:59 |
<Krinkle> |
mw1342 is generating strange PHP erros (php7 only), ref T224491 |
[production] |
| 00:58 |
<urandom> |
bootstrapping restbase1017-a -- T222960 |
[production] |
| 00:50 |
<mutante> |
restbase1018 - restart ferm service |
[production] |
| 00:15 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: e4bd91f71b (duration: 00m 50s) |
[production] |
| 00:13 |
<krinkle@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: f309856f0912 (duration: 00m 50s) |
[production] |
| 00:03 |
<eevans@deploy1001> |
Finished deploy [cassandra/metrics-collector@df909a1]: deploy logback to restbase1017 (T222960) (duration: 00m 03s) |
[production] |
| 00:03 |
<eevans@deploy1001> |
Started deploy [cassandra/metrics-collector@df909a1]: deploy logback to restbase1017 (T222960) |
[production] |
| 00:01 |
<eevans@deploy1001> |
Finished deploy [cassandra/metrics-collector@df909a1]: deploy logback to restbase1017 (T222960) (duration: 00m 25s) |
[production] |
| 00:01 |
<eevans@deploy1001> |
Started deploy [cassandra/metrics-collector@df909a1]: deploy logback to restbase1017 (T222960) |
[production] |
|
2019-07-11
§
|
| 23:58 |
<thcipriani@deploy1001> |
Synchronized php-1.34.0-wmf.13/includes/watcheditem/WatchedItemStore.php: SWAT: [[gerrit:522155|WatchedItemStore: Fix fatal when revision is deleted]] T226741 (duration: 00m 51s) |
[production] |
| 23:49 |
<eevans@deploy1001> |
Finished deploy [cassandra/logstash-logback-encoder@d085ffa]: deploy logback to restbase1017 (T222960) (duration: 00m 47s) |
[production] |
| 23:48 |
<eevans@deploy1001> |
Started deploy [cassandra/logstash-logback-encoder@d085ffa]: deploy logback to restbase1017 (T222960) |
[production] |
| 23:47 |
<eevans@deploy1001> |
Finished deploy [cassandra/logstash-logback-encoder@d085ffa]: (no justification provided) (duration: 01m 56s) |
[production] |
| 23:45 |
<eevans@deploy1001> |
Started deploy [cassandra/logstash-logback-encoder@d085ffa]: (no justification provided) |
[production] |
| 23:38 |
<eevans@deploy1001> |
deploy aborted: (no justification provided) (duration: 02m 00s) |
[production] |
| 23:36 |
<eevans@deploy1001> |
Started deploy [cassandra/logstash-logback-encoder@d085ffa]: (no justification provided) |
[production] |
| 23:15 |
<thcipriani@deploy1001> |
Synchronized wmf-config: SWAT: [[gerrit:521338|Oversample all EditAttemptStep events on VE-as-mobile-default wikis]] T227317 (duration: 00m 50s) |
[production] |
| 22:59 |
<mutante> |
netmon1003 - removing servermon - servermon.wikimedia.org is being decom'ed (T198939) |
[production] |