|
2016-10-06
ยง
|
| 21:12 |
<ostriches> |
lead: enabling & running puppet again, should bring things back up |
[production] |
| 21:02 |
<akosiaris> |
rebooting lead one more time |
[production] |
| 20:17 |
<akosiaris> |
rebooting lead once more |
[production] |
| 19:47 |
<ostriches> |
lead: rebooting, because what have we got to lose |
[production] |
| 19:23 |
<twentyafterfour> |
disabled phd and puppet on iridium |
[production] |
| 18:31 |
<twentyafterfour> |
stopped phd on iridium to relieve some load on gerrit |
[production] |
| 18:24 |
<ostriches> |
lead: restarting apache to force error page to show for now |
[production] |
| 18:21 |
<ostriches> |
lead: disabled puppet for now, gerrit's sick |
[production] |
| 18:04 |
<ostriches> |
gerrit: kicking gerrit and apache, something is unhappy... |
[production] |
| 17:11 |
<ema> |
power-cycling cp2017 |
[production] |
| 16:54 |
<akosiaris> |
uploaded to apt.wikimedia.org precise-wikimedia/main: php5_5.3.10-1ubuntu3.25+wmf1 |
[production] |
| 16:31 |
<ema> |
power-cycling cp2022 |
[production] |
| 15:48 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Oct 6 15:48:38 UTC 2016 (duration 7m 12s) |
[production] |
| 15:41 |
<dereckson@tin> |
scap sync-l10n completed (1.28.0-wmf.21) (duration: 15m 46s) |
[production] |
| 15:41 |
<_joe_> |
upgrading conftool to 0.3.1 on all mw*, wtp* servers, T147480 T145518 |
[production] |
| 15:25 |
<ema> |
powercycle cp3045 |
[production] |
| 15:05 |
<dereckson@tin> |
scap sync-l10n completed (1.28.0-wmf.20) (duration: 15m 59s) |
[production] |
| 15:03 |
<ema> |
cp3034 hanging during boot, power-cycled |
[production] |
| 14:41 |
<jynus> |
restarting db1069:3133 mysql instance |
[production] |
| 14:40 |
<_joe_> |
uploaded conftool 0.3.1 to apt.w.o, T147480 |
[production] |
| 14:33 |
<ema> |
cache_upload: rolling reboots for kernel upgrades |
[production] |
| 14:26 |
<dereckson@tin> |
Synchronized php-1.28.0-wmf.21/extensions/WikimediaMessages/i18n/wikimedia: Wikimedia messages for new 'engineer' group for ruwiki (T144599) (duration: 00m 49s) |
[production] |
| 14:25 |
<dereckson@tin> |
Synchronized php-1.28.0-wmf.20/extensions/WikimediaMessages/i18n/wikimedia: Wikimedia messages for new 'engineer' group for ruwiki (T144599) (duration: 00m 49s) |
[production] |
| 14:23 |
<dereckson@tin> |
Synchronized wmf-config/InitialiseSettings.php: New 'engineer' group for ruwiki (T144599) (duration: 00m 52s) |
[production] |
| 13:58 |
<dereckson@tin> |
Synchronized wmf-config/InitialiseSettings.php: Configure Visual Editor namespaces on sv.wikipedia ([[gerrit:309808]] and [[gerrit:314558]], T144688) (duration: 00m 50s) |
[production] |
| 13:54 |
<mobrovac> |
citoid deploying 4d97774 |
[production] |
| 13:50 |
<elukey> |
added mw122[67] back to the api appservers live pool |
[production] |
| 13:50 |
<ema> |
cache_text: rolling reboots for kernel upgrades |
[production] |
| 13:38 |
<dereckson@tin> |
Synchronized wmf-config/CommonSettings.php: Disable Upload Wizard blacklist issues on Commons (T146417) (duration: 00m 49s) |
[production] |
| 13:23 |
<dereckson@tin> |
Synchronized php-1.28.0-wmf.21/extensions/EventBus: Send a resource_change event on page_image property change (T145569) (duration: 00m 48s) |
[production] |
| 13:15 |
<dereckson@tin> |
Synchronized wmf-config/CirrusSearch-common.php: Initialize subphrases autocomplete on wikisources, mw.org and wikitech (T146208, 3/3) (duration: 00m 49s) |
[production] |
| 13:14 |
<dereckson@tin> |
Synchronized wmf-config/InitialiseSettings.php: Initialize subphrases autocomplete on wikisources, mw.org and wikitech (T146208, 2/3) (duration: 00m 49s) |
[production] |
| 13:11 |
<dereckson@tin> |
Synchronized tests/cirrusTest.php: Initialize subphrases autocomplete on wikisources, mw.org and wikitech (T146208, 1/3, no-op in prod part) (duration: 00m 50s) |
[production] |
| 12:50 |
<ema> |
cache_misc: rolling reboots for kernel upgrades |
[production] |
| 12:36 |
<elukey> |
reimaging mw122[67] to Debian Jessie |
[production] |
| 12:33 |
<elukey> |
adding mw122[45] back to the live api appservers pool (note: mw1224 was pooled => no before the reimage, but I don't see any blocker in adding it back to serve live traffic) |
[production] |
| 11:57 |
<mobrovac> |
restbase deploy end of fa4dc79 |
[production] |
| 11:50 |
<moritzm> |
rebooting video scalers for kernel security update |
[production] |
| 11:36 |
<mobrovac> |
restbase deploy start of fa4dc79 |
[production] |
| 11:14 |
<elukey> |
reimaging mw122[34] to Debian Jessie |
[production] |
| 11:12 |
<elukey> |
added mw122[23] back to the api appservers live pool |
[production] |
| 10:50 |
<mobrovac> |
change-prop deploying 403eec8 |
[production] |
| 10:27 |
<ema> |
cache_maps: rolling reboots for kernel upgrades |
[production] |
| 10:16 |
<moritzm> |
reimaging mw1209, mw1210, mw1215 to jessie |
[production] |
| 10:09 |
<ema> |
power cycling cp2015, reboot failed |
[production] |
| 10:02 |
<elukey> |
reimaging mw122[23] to Debian jessie (api appservers) |
[production] |
| 09:57 |
<ema> |
cp1046 cp2015 depooled reboot for kernel upgrades |
[production] |
| 09:55 |
<moritzm> |
installing jackrabbit security updates on Ubuntu and Debian systems |
[production] |
| 09:36 |
<elukey> |
adding mw1208 and mw1221 back to the api appservers live pool |
[production] |
| 09:22 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Restoring db1082 original weight: 500 (duration: 00m 52s) |
[production] |