|
2015-09-21
§
|
| 14:09 |
<godog> |
rolling restart restbase in production after cassandra credentials change |
[production] |
| 12:53 |
<godog> |
rolling restart cassandra after enabling dc encryption, no nodes in codfw yet |
[production] |
| 12:01 |
<moritzm> |
repooled mw1160 (for T104969) |
[production] |
| 11:54 |
<moritzm> |
depooled mw1160 (for T104969) |
[production] |
| 11:51 |
<moritzm> |
repooled mw1158, mw1159 (for T104969) |
[production] |
| 11:39 |
<moritzm> |
depooled mw1158, mw1159 (for T104969) |
[production] |
| 11:37 |
<moritzm> |
depooled and repooled mw1156, mw1157 (for T104969) |
[production] |
| 11:26 |
<moritzm> |
repooled mw1154, mw1155 (for T104969) |
[production] |
| 11:21 |
<moritzm> |
depooled mw1154, mw1155 (for T104969) |
[production] |
| 10:39 |
<moritzm> |
repooled mw1026-mw1029 and mw1110-mw1113 (for T104968) |
[production] |
| 10:24 |
<moritzm> |
depooled mw1026-mw1029 and mw1110-mw1113 (for T104968) |
[production] |
| 10:17 |
<moritzm> |
repooled mw1100-mw1109 (for T104968) |
[production] |
| 10:17 |
<godog> |
create restbase user on cassandra cluster |
[production] |
| 10:06 |
<moritzm> |
depooled mw1100-mw1109 (for T104968) |
[production] |
| 09:56 |
<moritzm> |
repooled mw1140 and mw1142-mw1148 (for T104968) |
[production] |
| 09:41 |
<moritzm> |
depooled mw1140 and mw1142-mw1148 (for T104968) |
[production] |
| 09:36 |
<moritzm> |
repooled mw1130-mw1139 (for T104968) |
[production] |
| 09:22 |
<moritzm> |
depooled mw1130-mw1139 (for T104968) |
[production] |
| 09:14 |
<moritzm> |
repooled mw1120-mw1129 (for T104968) |
[production] |
| 09:02 |
<moritzm> |
depooled mw1120-mw1129 (for T104968) |
[production] |
| 08:48 |
<moritzm> |
repooled mw1189 and mw1200-mw1208 (for T104968) |
[production] |
| 08:33 |
<moritzm> |
depooled mw1189 and mw1200-mw1208 (for T104968) |
[production] |
| 08:29 |
<godog> |
switch to 'restbase' cassandra user on restbase test cluster |
[production] |
| 08:29 |
<moritzm> |
repooled mw1190-mw1195 and mw1197-mw1199 (for T104968) |
[production] |
| 08:21 |
<_joe_> |
restarted the logstash agent on logstash1003, OOM'd |
[production] |
| 08:18 |
<moritzm> |
depooled mw1190-mw1195 and mw1197-mw1199 (for T104968) |
[production] |
| 08:07 |
<_joe_> |
installing the new HHVM package on the api canaries |
[production] |
| 08:04 |
<moritzm> |
repooled mw1221-mw1229 (for T104968) |
[production] |
| 07:53 |
<moritzm> |
depooled mw1221-mw1229 (for T104968) |
[production] |
| 07:49 |
<moritzm> |
repooled mw1230-mw1235 (for T104968) |
[production] |
| 07:43 |
<_joe_> |
installing the new hhvm package on the canary appservers |
[production] |
| 07:08 |
<moritzm> |
depooled mw1230-mw1235 (for T104968) |
[production] |
| 04:31 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Sep 21 04:31:03 UTC 2015 (duration 31m 2s) |
[production] |
| 02:23 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.26wmf23) at 2015-09-21 02:23:12+00:00 |
[production] |
| 02:19 |
<l10nupdate@tin> |
Synchronized php-1.26wmf23/cache/l10n: l10nupdate for 1.26wmf23 (duration: 06m 25s) |
[production] |
| 02:06 |
<MaxSem> |
Maps: created indexes on admin. <3 Postgres :( |
[production] |
| 01:56 |
<bblack> |
downtimed eqiad ipv6 text/upload alerts as well, as with mobile above ( 1 301 TLS Redirect - 505 bytes in 1.008 second response time |
[production] |
| 01:46 |
<bblack> |
downtimed the "LVS HTTP IPv6 on mobile-lb.eqiad.wikimedia.org_ipv6" alert for now ( https://phabricator.wikimedia.org/T113154 ) |
[production] |