|
2014-11-05
§
|
| 18:02 |
<^d> |
elastic1022 unbanned from allocation since it has a network cable again |
[production] |
| 17:14 |
<demon> |
Synchronized wmf-config/PoolCounterSettings-eqiad.php: (no message) (duration: 00m 06s) |
[production] |
| 17:02 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: frwiki getting cirrusy search (duration: 00m 05s) |
[production] |
| 10:47 |
<_joe_> |
installed hhvm 3.3.0-20140925+wmf4 on osmium for testing. |
[production] |
| 09:11 |
<akosiaris> |
depool wtp1002, wtp1007-wtp1012 |
[production] |
| 09:09 |
<akosiaris> |
repool wtp1013,wtp1014,wtp1015,wtp1016,wtp1017 |
[production] |
| 07:18 |
<ori> |
rolled back cluster:appserver_hhvm to version 3.3.0-20140925+wmf3 of hhvm package |
[production] |
| 06:29 |
<akosiaris> |
depool wtp1013, wtp1014, wtp1015, wtp1016, wtp1023 for trusty reinstallation |
[production] |
| 05:53 |
<ori> |
ran: salt -G php:hhvm cmd.run 'restart hhvm' |
[production] |
| 05:26 |
<ori> |
Synchronized wmf-config/CommonSettings.php: $wgPercentHHVM: back to 5% (duration: 00m 11s) |
[production] |
| 03:21 |
<^d|voted> |
restarted lucene-search-2 on search1019: it'd been timing out for a few days and filled disk with log files. |
[production] |
| 02:25 |
<ori> |
Synchronized wmf-config/CommonSettings.php: If866e9caf: $wgPercentHHVM: 5 => 10, to test https://phabricator.wikimedia.org/T820#18870 (duration: 00m 04s) |
[production] |
| 01:21 |
<ori> |
Synchronized php-1.25wmf5/extensions/MobileFrontend: Ic82ba72b98: Update MobileFrontend for cherry-picks (duration: 00m 04s) |
[production] |
| 01:21 |
<ori> |
Synchronized php-1.25wmf6/extensions/MobileFrontend: Ic26f56c0d: Update MobileFrontend for cherry-picks (duration: 00m 05s) |
[production] |
| 01:12 |
<^d|voted> |
elastic1022: banned from allocation since its unreachable. just in case it starts flapping. |
[production] |
| 01:11 |
<mutante> |
elatic1022 - eth0: <NO-CARRIER |
[production] |
| 01:07 |
<ori> |
upgrading HHVM app servers to 3.3.0+dfsg1-1+wm2 |
[production] |
| 01:02 |
<mutante> |
powercycling elastic1022 |
[production] |
| 00:45 |
<^d|voted> |
elasticsearch: rebuilding all cirrus indexes for all wikis from a screen on terbium, going to take awhile. should be boring, but if causing problems kill it first and then find me. |
[production] |
| 00:24 |
<demon> |
Synchronized php-1.25wmf6/includes/parser/Parser.php: (no message) (duration: 00m 04s) |
[production] |
| 00:23 |
<demon> |
Synchronized php-1.25wmf6/includes/parser/CoreTagHooks.php: (no message) (duration: 00m 04s) |
[production] |
| 00:23 |
<demon> |
Synchronized php-1.25wmf5/includes/parser/Parser.php: (no message) (duration: 00m 04s) |
[production] |
| 00:23 |
<demon> |
Synchronized php-1.25wmf5/includes/parser/CoreTagHooks.php: (no message) (duration: 00m 05s) |
[production] |
| 00:22 |
<demon> |
Synchronized php-1.25wmf6/extensions/CirrusSearch/maintenance/updateOneSearchIndexConfig.php: (no message) (duration: 00m 04s) |
[production] |
| 00:22 |
<demon> |
Synchronized php-1.25wmf5/extensions/CirrusSearch/maintenance/updateOneSearchIndexConfig.php: (no message) (duration: 00m 04s) |
[production] |
| 00:04 |
<demon> |
Synchronized wmf-config/CirrusSearch-common.php: (no message) (duration: 00m 04s) |
[production] |
| 00:03 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 07s) |
[production] |
|
2014-11-04
§
|
| 19:22 |
<cmjohnson> |
rebooting wtp1023 |
[production] |
| 17:21 |
<ejegg> |
updated crm from b8a1fa98b5d9252d708090c99b61fd22ebe8d2be to e9e81a828d50e8bddf98eae699c925e09b25927b |
[production] |
| 16:53 |
<akosiaris> |
repool wtp1017,wtp1018,wtp1019,wtp1020 |
[production] |
| 16:50 |
<hashar> |
restarting Zuul/Jenkins entirely |
[production] |
| 16:45 |
<manybubbles> |
Synchronized php-1.25wmf6/extensions/UniversalLanguageSelector/: SWAT update uls (duration: 00m 04s) |
[production] |
| 16:43 |
<manybubbles> |
Synchronized php-1.25wmf5/extensions/UniversalLanguageSelector/: SWAT update uls (duration: 00m 04s) |
[production] |
| 16:33 |
<hashar> |
Shutting down Jenkins to remove a deadlock :-( |
[production] |
| 16:27 |
<hashar> |
Jenkins restarting Gearman client |
[production] |
| 16:24 |
<hashar> |
Zuul on hold, waiting for beta cluster related jobs to complete |
[production] |
| 16:21 |
<hashar> |
Jenkins: disconnecting/reconnecting gearman client , killing deployment-bastion.eqiad slave in an attempt to remove a deadlock {{bug|70597}} |
[production] |
| 14:06 |
<akosiaris> |
upgrading kernels on amssq* |
[production] |
| 13:59 |
<reedy> |
Synchronized wmf-config/: (no message) (duration: 00m 14s) |
[production] |
| 13:58 |
<Reedy> |
graceful apache on mw1193 |
[production] |
| 13:57 |
<Reedy> |
graceful apache on mw1144 |
[production] |
| 13:51 |
<reedy> |
Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 15s) |
[production] |
| 13:40 |
<akosiaris> |
depool wtp1017, wtp1018, wtp1019, wtp1020 from trusty reinstall |
[production] |
| 13:39 |
<akosiaris> |
upgrading apache2 throught the mw cluster |
[production] |
| 13:24 |
<reedy> |
Synchronized wmf-config/: (no message) (duration: 00m 19s) |
[production] |
| 13:14 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.25wmf6 |
[production] |
| 13:03 |
<reedy> |
Purged l10n cache for 1.25wmf4 |
[production] |
| 12:41 |
<akosiaris> |
repooled wtp1021,wtp1022,wtp1023 |
[production] |
| 10:42 |
<akosiaris> |
depooled wtp1021,wtp1022,wtp1023 for re-installation with trusty |
[production] |
| 06:31 |
<springle> |
force logrotate ocg1001 |
[production] |