|
2018-10-10
ยง
|
| 18:05 |
<otto@deploy1001> |
Started deploy [analytics/refinery@4e2d956]: Add accept header to webrequest logs - T170606 |
[production] |
| 17:49 |
<XioNoX> |
replace 10.195.0.0/25 with 10.195.0.0/24 in prefix-list fundraising-codfw4 on cr1/2-codfw - T206637 |
[production] |
| 16:25 |
<mutante> |
LDAP - added isaacj to wmf group (for SWAP access, existing shell user since recently) (T206631) (T205840) |
[production] |
| 16:16 |
<_joe_> |
restart of now-unused jobqueue redises for stopping the alerts post-switchover |
[production] |
| 16:09 |
<ejegg> |
updated CiviCRM from 1165e7ed79 to 4cc21d61c5 |
[production] |
| 15:59 |
<vgutierrez> |
Uploaded certcentral 0.1 to apt.wikimedia.org (stretch) - T199711 |
[production] |
| 15:55 |
<cmjohnson1> |
scheduled downtime for host cloudvirt1019 swap raid card T196507 |
[production] |
| 15:35 |
<moritzm> |
uploaded jenkins 2.138.2 security release to apt.wikimedia.org (jessie/stretch) (T206234) |
[production] |
| 15:11 |
<_joe_> |
started again hhvm on mwmaint2001 |
[production] |
| 14:51 |
<ejegg> |
turned fundraising scheduled jobs back on |
[production] |
| 14:43 |
<ejegg> |
turned off fundraising scheduled jobs |
[production] |
| 14:42 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.08-restore-ttl (exit_code=0) (volans@neodymium) |
[production] |
| 14:42 |
<START> |
- Cookbook sre.switchdc.mediawiki.08-restore-ttl (volans@neodymium) |
[production] |
| 14:42 |
<END> |
(FAIL) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=99) (volans@neodymium) |
[production] |
| 14:40 |
<START> |
- Cookbook sre.switchdc.mediawiki.08-start-maintenance (volans@neodymium) |
[production] |
| 14:39 |
<END> |
(FAIL) - Cookbook sre.switchdc.mediawiki.08-start-maintenance (exit_code=99) (volans@neodymium) |
[production] |
| 14:38 |
<START> |
- Cookbook sre.switchdc.mediawiki.08-start-maintenance (volans@neodymium) |
[production] |
| 14:33 |
<oblivian@puppetmaster1001> |
conftool action : set/weight=15; selector: cluster=api_appserver,service=apache2,dc=eqiad,name=mw123.* |
[production] |
| 14:31 |
<oblivian@puppetmaster1001> |
conftool action : set/weight=15; selector: cluster=api_appserver,service=apache2,dc=eqiad,name=mw122.* |
[production] |
| 14:19 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.08-update-tendril (exit_code=0) (volans@neodymium) |
[production] |
| 14:19 |
<START> |
- Cookbook sre.switchdc.mediawiki.08-update-tendril (volans@neodymium) |
[production] |
| 14:18 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.07-set-readwrite (exit_code=0) (volans@neodymium) |
[production] |
| 14:18 |
<MediaWiki> |
read-only period ends at: 2018-10-10 14:18:26.908958 (volans@neodymium) |
[production] |
| 14:18 |
<START> |
- Cookbook sre.switchdc.mediawiki.07-set-readwrite (volans@neodymium) |
[production] |
| 14:18 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (exit_code=0) (volans@neodymium) |
[production] |
| 14:18 |
<START> |
- Cookbook sre.switchdc.mediawiki.06-set-db-readwrite (volans@neodymium) |
[production] |
| 14:17 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (exit_code=0) (volans@neodymium) |
[production] |
| 14:17 |
<START> |
- Cookbook sre.switchdc.mediawiki.05-invert-redis-sessions (volans@neodymium) |
[production] |
| 14:17 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.04-switch-traffic (exit_code=0) (volans@neodymium) |
[production] |
| 14:15 |
<START> |
- Cookbook sre.switchdc.mediawiki.04-switch-traffic (volans@neodymium) |
[production] |
| 14:15 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (exit_code=0) (volans@neodymium) |
[production] |
| 14:14 |
<START> |
- Cookbook sre.switchdc.mediawiki.04-switch-mediawiki (volans@neodymium) |
[production] |
| 14:14 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.03-set-db-readonly (exit_code=0) (volans@neodymium) |
[production] |
| 14:14 |
<START> |
- Cookbook sre.switchdc.mediawiki.03-set-db-readonly (volans@neodymium) |
[production] |
| 14:14 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.02-set-readonly (exit_code=0) (volans@neodymium) |
[production] |
| 14:13 |
<MediaWiki> |
read-only period starts at: 2018-10-10 14:13:46.068081 (volans@neodymium) |
[production] |
| 14:13 |
<START> |
- Cookbook sre.switchdc.mediawiki.02-set-readonly (volans@neodymium) |
[production] |
| 14:10 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.01-stop-maintenance (exit_code=0) (volans@neodymium) |
[production] |
| 14:10 |
<START> |
- Cookbook sre.switchdc.mediawiki.01-stop-maintenance (volans@neodymium) |
[production] |
| 14:10 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.00-warmup-caches (exit_code=0) (volans@neodymium) |
[production] |
| 14:07 |
<START> |
- Cookbook sre.switchdc.mediawiki.00-warmup-caches (volans@neodymium) |
[production] |
| 14:07 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.00-warmup-caches (exit_code=0) (volans@neodymium) |
[production] |
| 14:05 |
<START> |
- Cookbook sre.switchdc.mediawiki.00-warmup-caches (volans@neodymium) |
[production] |
| 14:05 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.00-warmup-caches (exit_code=0) (volans@neodymium) |
[production] |
| 14:01 |
<START> |
- Cookbook sre.switchdc.mediawiki.00-warmup-caches (volans@neodymium) |
[production] |
| 14:01 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.00-reduce-ttl (exit_code=0) (volans@neodymium) |
[production] |
| 14:01 |
<START> |
- Cookbook sre.switchdc.mediawiki.00-reduce-ttl (volans@neodymium) |
[production] |
| 14:00 |
<END> |
(PASS) - Cookbook sre.switchdc.mediawiki.00-disable-puppet (exit_code=0) (volans@neodymium) |
[production] |
| 14:00 |
<START> |
- Cookbook sre.switchdc.mediawiki.00-disable-puppet (volans@neodymium) |
[production] |
| 12:18 |
<_joe_> |
decommissioning conf1001-1003: stopping etcd, nginx, and masking both |
[production] |