|
2013-03-26
ยง
|
| 22:10 |
<notpeter> |
icinga mostly trustworthy again :) |
[production] |
| 22:08 |
<andrewbogott> |
restarted nova-api on virt0 because it appeared to have crashed. |
[production] |
| 21:31 |
<andrewbogott> |
restarted memcached on virt0 in a futile attempt to address https://bugzilla.wikimedia.org/show_bug.cgi?id=46583 |
[production] |
| 21:29 |
<mutante> |
killed stuck snmptt procs on neon to fix the others.. apparently :p |
[production] |
| 21:28 |
<mutante> |
stopping gmetad on neon |
[production] |
| 20:51 |
<kaldari> |
synchronized wmf-config/CommonSettings.php 'syncing CommonSettings.php for Echo event tracking' |
[production] |
| 20:41 |
<mutante> |
more degraded RAIDs: db1001->RT-4832, db1028->4834 |
[production] |
| 20:35 |
<mutante> |
broken disks/degraded RAID on db servers: db11->RT-4828, db29->RT-4829, db45->RT-4831 |
[production] |
| 20:34 |
<kaldari> |
synchronized php-1.21wmf12/extensions/ContributionReporting 'syncing ContributionReporting extension' |
[production] |
| 20:07 |
<asher> |
synchronized wmf-config/db-eqiad.php 'raising db1051 to full weight' |
[production] |
| 19:49 |
<khorn> |
synchronized php-1.21wmf12/extensions/CentralNotice 'Fixing CentralNotice bug in which it was possible to initialize multiple times' |
[production] |
| 19:45 |
<khorn> |
synchronized php-1.21wmf11/extensions/CentralNotice 'Fixing CentralNotice bug in which it was possible to initialize multiple times' |
[production] |
| 19:40 |
<kaldari> |
synchronized php-1.21wmf12/resources/startup.js 'syncing php-1.21wmf12/resources/startup.js' |
[production] |
| 19:32 |
<reedy> |
synchronized php-1.21wmf12/extensions/Wikibase/client |
[production] |
| 19:27 |
<cmjohnson1> |
mw1209 re-imaging |
[production] |
| 19:20 |
<kaldari> |
synchronized php-1.21wmf12/extensions/Echo/modules/icons 're-syncing Echo icons dir' |
[production] |
| 19:19 |
<mlitn> |
synchronized wmf-config/InitialiseSettings.php 'Include AFTv5 on frwiki' |
[production] |
| 19:12 |
<mlitn> |
synchronized wmf-config 'Enable AFTv5 on frwiki' |
[production] |
| 19:05 |
<reedy> |
synchronized php-1.21wmf12/extensions/Collection/ |
[production] |
| 19:04 |
<mlitn> |
synchronized php-1.21wmf12/extensions/ArticleFeedbackv5 'Update ArticleFeedbackv5 to master' |
[production] |
| 19:03 |
<mlitn> |
synchronized php-1.21wmf11/extensions/ArticleFeedbackv5 'Update ArticleFeedbackv5 to master' |
[production] |
| 19:02 |
<kaldari> |
synchronized php-1.21wmf12/extensions/Echo/modules/icons 'syncing Echo icons dir' |
[production] |
| 18:58 |
<mutante> |
starting gmetad on neon - we failed to resolve data source name ms-fe1001.eqiad.wmnet |
[production] |
| 18:56 |
<mutante> |
restarting ircecho on fenari |
[production] |
| 18:48 |
<mutante> |
stopping gmetad on neon |
[production] |
| 18:44 |
<mutante> |
killing all snmp related processes on neon and restarting |
[production] |
| 18:43 |
<reedy> |
synchronized php-1.21wmf12/extensions/MobileFrontend/includes/specials/SpecialMobileWatchlist.php |
[production] |
| 18:43 |
<reedy> |
synchronized php-1.21wmf12/extensions/MobileFrontend/includes/specials/SpecialMobileWatchlist.php |
[production] |
| 18:41 |
<kaldari> |
synchronized php-1.21wmf11/extensions/Echo/modules/icons 'syncing Echo icons dir' |
[production] |
| 18:41 |
<kaldari> |
synchronized php-1.21wmf11/extensions/Echo/modules/icons 'syncing Echo icons dir' |
[production] |
| 18:38 |
<mutante> |
restarting snmtrapd on neon |
[production] |
| 18:25 |
<mlitn> |
synchronized wmf-config/InitialiseSettings.php 'Turning Thanks extension on on MediaWiki wiki' |
[production] |
| 18:24 |
<mlitn> |
synchronized wmf-config/InitialiseSettings.php 'Turning Thanks extension on on MediaWiki wiki' |
[production] |
| 18:13 |
<krinkle> |
synchronized php-1.21wmf12/resources/mediawiki/mediawiki.js 'I80af730daa815 fixing bug 46575' |
[production] |
| 18:13 |
<krinkle> |
synchronized php-1.21wmf12/resources/mediawiki/mediawiki.js 'I80af730daa815 fixing bug 46575' |
[production] |
| 18:12 |
<Krinkle> |
Change I34a488247 (mw.loader debug) has been reverted in wmf/1.21wmf12 and pulled on fenari |
[production] |
| 17:51 |
<cmjohnson1> |
removing disk from slot 0 ms-be1012 |
[production] |
| 17:14 |
<Krinkle> |
Change I34a488247 is now merged in wmf/1.21wmf12 and pulled on fenari for test.wikipedia.org, do not sync. |
[production] |
| 17:04 |
<ottomata> |
added (crappy, but better than nothing) .deb for webstatscollector to apt.wikimedia.org |
[production] |
| 15:24 |
<cmjohnson1> |
swapping disk 6 ms-be1012 |
[production] |
| 14:34 |
<hashar> |
synchronized multiversion '{{gerrit|47564}} wikiversions.cdb now vary by realm' |
[production] |
| 14:34 |
<hashar> |
synchronized multiversion '{{gerrit|47564}} wikiversions.cdb now vary by realm' |
[production] |
| 14:09 |
<cmjohnson1> |
running sync-common on mw1209 |
[production] |
| 10:23 |
<nikerabbit> |
synchronized php-1.21wmf12/extensions/UniversalLanguageSelector/UniversalLanguageSelector.php 'Testing fix for wikidata' |
[production] |
| 10:23 |
<nikerabbit> |
synchronized php-1.21wmf12/extensions/UniversalLanguageSelector/UniversalLanguageSelector.php 'Testing fix for wikidata' |
[production] |
| 09:28 |
<pp-pdf1> |
updated mwlib to 0.15.4 |
[production] |
| 08:47 |
<nikerabbit> |
synchronized php-1.21wmf12/extensions/Translate |
[production] |
| 08:47 |
<nikerabbit> |
synchronized php-1.21wmf12/extensions/Translate |
[production] |
| 08:46 |
<nikerabbit> |
synchronized php-1.21wmf12/extensions/UniversalLanguageSelector |
[production] |
| 08:46 |
<nikerabbit> |
synchronized php-1.21wmf12/extensions/UniversalLanguageSelector |
[production] |