|
2021-06-29
§
|
| 08:46 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw1355.eqiad.wmnet |
[production] |
| 08:43 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: name=elastic1039.eqiad.wmnet |
[production] |
| 08:25 |
<elukey> |
cumin 'A:mw-eqiad' '/usr/local/sbin/restart-php7.2-fpm' -b 2 -s 30 - T285634 |
[production] |
| 08:21 |
<elukey> |
depool mw1355 (mw appserver) for debugging - T285634 |
[production] |
| 08:21 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=mw1355.eqiad.wmnet |
[production] |
| 08:12 |
<hashar> |
Upgrading Jenkins on contint2001 / contint1001 and restarting CI Jenkins # T285531 |
[production] |
| 08:03 |
<hashar> |
Upgraded Jenkins on releases1002 / releases2002 # T285531 |
[production] |
| 08:02 |
<hashar> |
Upgraded Jenkins on releases1002 / releases2002 |
[production] |
| 07:49 |
<godog> |
remove 20G migration data /root/prometheus from prometheus4001 - T243057 |
[production] |
| 07:48 |
<godog> |
remove old /root/prometheus data from prometheus4001 |
[production] |
| 07:05 |
<moritzm> |
upgrading bullseye early installs to the latest state of testing T275873 |
[production] |
| 06:46 |
<tstarling@deploy1002> |
Synchronized php-1.37.0-wmf.11/includes/MediaWiki.php: Add statsd action timing metric T284274 (duration: 00m 58s) |
[production] |
| 02:47 |
<cdanis> |
✔️ cdanis@cumin2001.codfw.wmnet ~ 🕥🍺 sudo cumin -b16 'A:cp-upload and A:codfw' 'run-puppet-agent -q' |
[production] |
| 02:34 |
<ryankemper> |
T285643 Banned `elastic1039` from all 3 elasticsearch clusters and set `elastic1039.eqiad.wmnet` to failed in netbox |
[production] |
| 02:27 |
<cdanis> |
✔️ cdanis@cumin2001.codfw.wmnet ~ 🕥🍺 sudo cumin -b16 'A:cp-upload' 'run-puppet-agent -q' |
[production] |
| 02:25 |
<eileen> |
civicrm revision changed from 927ab7cff7 to 789c92d13b, config revision is 1739c53fcb |
[production] |
| 02:04 |
<ryankemper@deploy1002> |
Finished deploy [wdqs/wdqs@0e916b1]: 0.3.75 (duration: 08m 40s) |
[production] |
| 01:57 |
<ryankemper> |
[WDQS Deploy] Tests passing following deploy of `0.3.75` on canary `wdqs1003`; proceeding to rest of fleet |
[production] |
| 01:56 |
<ryankemper@deploy1002> |
Started deploy [wdqs/wdqs@0e916b1]: 0.3.75 |
[production] |
| 01:50 |
<ryankemper> |
[WDQS Deploy] Gearing up for deploy of wdqs `0.3.75`. Pre-deploy tests passing on canary `wdqs1003` |
[production] |
| 00:25 |
<Krinkle> |
krinkle@mwmaint1002: purgeParserCache.php --tag pc1, ref T282761 |
[production] |
|
2021-06-28
§
|
| 23:07 |
<urbanecm> |
Evening B&C window done |
[production] |
| 23:05 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 5ec855d14b31a9392274c2bfe2e21e2ad44986bc: Enable Parsoid inspired media structure on test wikis (T51097) (duration: 00m 59s) |
[production] |
| 22:51 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.08-update-tendril (exit_code=0) |
[production] |
| 22:51 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-update-tendril |
[production] |
| 22:50 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=0) |
[production] |
| 22:48 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-start-maintenance |
[production] |
| 22:48 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.08-run-puppet-on-db-masters (exit_code=0) |
[production] |
| 22:44 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-run-puppet-on-db-masters |
[production] |
| 22:43 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=0) |
[production] |
| 22:43 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-restore-ttl |
[production] |
| 22:43 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.08-restart-envoy-on-jobrunners (exit_code=0) |
[production] |
| 22:43 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.08-restart-envoy-on-jobrunners |
[production] |
| 22:43 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) |
[production] |
| 22:43 |
<legoktm@cumin1001> |
[DRY-RUN] MediaWiki read-only period ends at: 2021-06-28 22:43:04.512602 |
[production] |
| 22:43 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.07-set-readwrite |
[production] |
| 22:43 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) |
[production] |
| 22:42 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite |
[production] |
| 22:42 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (exit_code=0) |
[production] |
| 22:42 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions |
[production] |
| 22:42 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (exit_code=0) |
[production] |
| 22:42 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki |
[production] |
| 22:42 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.03-set-db-readonly (exit_code=0) |
[production] |
| 22:41 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.03-set-db-readonly |
[production] |
| 22:41 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.02-set-readonly (exit_code=0) |
[production] |
| 22:41 |
<legoktm@cumin1001> |
[DRY-RUN] MediaWiki read-only period starts at: 2021-06-28 22:41:41.222740 |
[production] |
| 22:41 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.02-set-readonly |
[production] |
| 22:40 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) |
[production] |
| 22:40 |
<legoktm@cumin1001> |
START - Cookbook sre.switchdc.mediawiki.01-stop-maintenance |
[production] |
| 22:40 |
<legoktm@cumin1001> |
END (PASS) - Cookbook sre.switchdc.mediawiki.00-warmup-caches (exit_code=0) |
[production] |